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INTRODUCTION 


College is a time of academic discovery and exploration. For many of the 16.8 million students enrolled at America’s 
two- and four-year degree programs each year, the university experience promotes academic growth, fosters new 
friendships, and expands understanding of a world outside the home environment. Approximately 75% of students 
attend four-year residential colleges and universities full time, which means the majority of these young adults are 
living away from home for the first time. 1 

In popular culture, the American college experience almost always includes drug or alcohol misuse as a rite of 
passage. However, despite the widespread use of alcohol and drugs in movies and television shows set on college 
campuses, these media reinforce a false narrative, especially when it comes to drug use. While almost 75% of 
college students report consuming alcohol at least once while in high school, drug use among college students tends 
to start while in college. 2 For savvy prevention professionals, the campus environment offers a unique opportunity 
to prevent the initiation of drug use among college students, the consequences of which can be long-lasting and 
devastating. 



Students often cite FOUR MAIN REASONS that 
college campuses provide a rich environment for 
drug experimentation: 

1. Ease of drug availability 

2. Lack of parental influence 

3. Normalization of drug use among peers 

4. Low perceived risk of harm from drug use 


Students often cite four main reasons that college campuses provide a rich environment for drug experimentation: 
(1) ease of drug availability, (2) lack of parental influence, (3) normalization of drug use among peers, and (4) low 
perceived risk of harm from drug use. 3 However, for college students who engage in drug use, the personal and 
academic costs can be high, even more for drug use than for alcohol use, leading to gaps in enrollment, prolonged 
time to graduation, and even failure to graduate. 4 Numerous studies have found an inverse relationship between 
consuming drugs intended to treat attention deficit and hyperactivity disorders (ADHD) (e.g., Adderall and Ritalin) 
as study aids and academic success. 5 In other words, nonmedical use of prescription stimulants does not improve 
academic performance. For a small minority of students, college drug experimentation leads to lifelong struggles 
with addiction. 6 


SO WHAT CAN BE DONE? 

Although the prevention field has spent the last 25 years understanding the complex nature of alcohol misuse on 
college campuses and creating campus-wide interventions, drug use remains, for many college health and wellness 
professionals, an individual issue, despite emerging evidence that the college environment contributes to the 
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INTRODUCTION 


initiation of drug use for the majority of college drug users. 7 College is the ideal setting for innovative, campus-wide 
programming aimed at preventing and reducing drug use among college students, but these efforts remain few and 
far between. 

This guide is intended to bridge that gap, by providing a road map for university prevention professionals to 
collaborate with a wide range of stakeholders, from students to administrators, to address campus-wide drug 
misuse issues. We use the Strategic Prevention Framework (SPF) here as the “how to” for systematically measuring 
the scope of drug misuse issues, building relationships with key stakeholders, and planning and implementing a 
drug misuse prevention effort. Developed by the Substance Abuse and Mental Health Services Administration in 
2004, the SPF is evidence based, widely used, and easily adaptable for multiple health issues. 


HOWTO USE THIS GUIDE 

CHAPTER 1 explores the history of drug misuse from the 1960s until today, 
and how changes in culture, federal and state laws and enforcement, and 
societal norms affect the prevention landscape on college campuses today. 

We will discuss the most commonly used drugs on college campuses and 
explore new and emerging drug trends. Finally, we focus on how the college 
environment can both protect and promote drug misuse and learn about federal 
policies that affect how colleges can address drug use. 

CHAPTER 2 provides an overview of the SPF, with a focus on cultural 
competence and cultural humility, two principles that underlie successful 
prevention work. We will also introduce the concept of sustainability, or 
the process of building an adaptive and effective system that achieves and 
maintains desired long-term results. 



CHAPTER 3 dives into the SPF’s first step, teaching how to assess the scope of drug use on your campus. We will 
explore primary and secondary effects of drug misuse and learn about risk and protective factors for individual 
students and at the campus level. We will also discuss how you can find data sources to assess the prevalence of 
drug misuse on your campus. 



CHAPTER 4 focuses on how to build capacity for a drug misuse prevention 
program. We will help you think strategically about your campus’s stakeholders 
and which groups of people you need to connect with to ensure your program is 
successful. We will discuss how to find champions to promote your drug misuse 
prevention program and allies to collaborate with you. We will also explore how 
much work you need to do to spotlight issues of drug misuse: How much do 
people know about this issue, and how can you figure that out? 

CHAPTER 5 centers on how to plan a drug misuse prevention program. We 
will show you how to prioritize your campus’s risk and protective factors and 
how to determine the long- and short-term outcomes for your program. We 
will highlight methods for selecting an evidence-based substance misuse 
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prevention program that meets your campus’s needs and how to adapt a 
program for your student population. We will provide steps to create a plan of 
action for your prevention program. 

CHAPTER 6 specifies how to implement a drug misuse prevention program. 

We will revisit your list of stakeholders and determine what roles they will play in 
your program. We will consider how to create a consistent program that is able 
to adapt as needed. We will help you think about how you can keep program 
champions in the loop during program planning and implementation. 

CHAPTER 7 provides tools to evaluate a prevention program to determine 
whether it was implemented as planned and whether it is making a difference 
in anticipated outcomes. We will discuss the differences between process 
and outcome evaluations and figure out how to determine which evaluation 
instruments can be adapted to assess your program. Lastly, we will think about 
which stakeholders should know about your evaluation results, and publicize success to the campus community. 

CHAPTER 8 offers guidance from Dolores Cimini of the University at Albany, a leader in the field of college 
substance misuse prevention, who provides lessons learned for both relative newcomers to the field and those 
prevention professionals who have been working on campus for five or more years. 

Finally, in APPENDIX A, we offer a list of resources and agencies to guide and inform your drug misuse prevention 
efforts. In APPENDIX B, we provide you with worksheets and tools you can use to work through the SPF’s steps with 
your prevention team. We reference tools throughout the text at appropriate points in the planning process when 
you might use them. The tools in the back are also available in PDF format at www.campusdrugprevention.gov . 

Let’s get started! 



How to Implement a Successful Drug Misuse 
Prevention Program on Your Campus 


a lot of freedom within those strategies. We want them to implement things that 
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One of the most powerful counterweights to substance use and one of the most 
powerful prevention strategies is to provide students with the challenging and 
rewarding experiences that a college education can provide. If students become very 
engaged and passionate and rewarded by something that is challenging, they might 
not have the time to be distracted by something like substance use that can take 
them off their path. 

—Dr. Amelia M. Arria, Director of the Center on Young Adult Health and Development and the Office of Planning and 
Evaluation at the University of Maryland School of Public Health 






CHAPTER 1: Drug Misuse Prevention Landscape 


History of Drug Legislation and How It Affects Current Campus 
Drug Misuse Prevention 

As Americans, we have a rich pop culture history linking college students and drug use. This association is based on 
historical fact. In the 1960s, the nation saw an unprecedented rise in illicit drug use among young people, including 
marijuana and LSD and other psychedelics. At the time, America was deeply divided, with generational differences 
not only on drug use, but also on the Vietnam War, sexual ethics, and the Civil Rights Movement. 8 

Perhaps the longest lasting change in social norms, however, was around the use of drugs for recreational use. 
Young people, driven by college students, were engaging in recreational drug use in increasing numbers. Lawmakers 
were forced to confront the efficacy of a previously punitive-based system of dealing with illicit drug use, which they 
were finding to be increasingly unproductive. With over 50 pieces of legislation to deal with various drug violations, 
the system was also incredibly unwieldy. 

In response, in 1970 Congress enacted the Comprehensive Drug Abuse and Control Act. Commonly known as the 
Controlled Substances Act (CSA), this law created the five-schedule system of drug classification that is still used 
now, where drugs are categorized according to their safety and medical utility. 9 The CSA also for the first time 
established a single system of control for narcotic and psychotropic drugs, overseen by what would eventually 
become the Drug Enforcement Administration. 10 

Most importantly to the prevention work conducted on college campuses today, the CSA also mandated the 
creation of the National Commission on Marihuana 1 and Drug Abuse. The NCMDA was charged with deepening 
understanding on the causes and effects of marijuana use, as well as providing recommendations to lawmakers on 
how to address the growing normalization of marijuana use across the nation. 

Within two reports written between 1971 and 1973, the NCMDA recommended the government fund research 
into the etiology and pharmacology of drug use, including marijuana and psychedelics, and continue the surveys 
of drug use and norms that they began. Notably, the NCMDA’s first report (1971) also strongly advocated for the 
decriminalization of marijuana for users (not dealers). 11 

The decriminalization recommendation was roundly rejected by President Nixon and set up what continues to be 
a national debate on how to deal with drug use among young people. From Reagan’s War on Drugs that led to the 
incarceration of young brown and black men in the 1980s for crack cocaine-related offenses to the current push 
for leniency in cases of opioid overdose to widespread public support for medical and retail cannabis legalization in 
states across the nation, the United States has struggled to define dangerous drug use, who should be punished for 
it, and what should be done about it. 12,13 ’ 14 

This history colors the current landscape we find ourselves in as health and wellness professionals on college 
campuses. We must understand this historical context to realize where resistance to our drug misuse prevention 
work comes from and how to effectively advocate for safer and healthier campus communities. 


The spelling marihuana was popularized in the 1920s by prohibitionists who sought to make cannabis (which until then was the preferred term) sound 
foreign and dangerous. It was used widely until shortly after the passage of the Controlled Substances Act, when the marijuana spelling gained in popularity. 
Today, the growing medical and retail markets increasingly prefer the term cannabis, which refers to the plant itself and does not carry the racist history of 
marihuana/marijuana. Ingraham, C. (2016, December 16). 'Marijuana’ or 'marihuana'? It’s all weed to the DEA. The Washington Post. 
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CHAPTER 1: Drug Misuse Prevention Landscape 


Current Trends in Young Adult Drug Use 


In addition to understanding history, we must also have an evidence-based perspective on how young people are 
currently using drugs and alcohol. There are differences in 30-day prevalence of drug use between young people in 
college and those of the same age who are currently not in college. 


Figure 1. College/non-college student past 30-Day drug use (2018) 15 


70 



Alcohol Five or Flavored Cannabis Cigarettes Vaping Vaping Vaping Cocaine Hallucinogens Opioids Adderall 
more drinks alcohol (any) Nicotine Cannabis (past year) (past year) 

in a row beverages 


Source: Schulenberg, J. E., Johnston, L. D., O’Malley, P. M., Bachman, J. G., Miech, R. A. & Patrick, M. E. (2019). Monitoring the Future national survey 
results on drug use , 1975-2018: Volume H, College students and adults ages 19-60. Ann Arbor, Ml: University of Michigan, Institute for Social 
Research. Retrieved from http://monitoringthefuture.Org/pubs.html#monographs 


How Race/Ethnicity Impacts Alcohol and Drug Use by College Students 


White students in general are more likely to engage in higher risk alcohol and other drug use than 
students of other ethnicities. White men, in particular, use alcohol and other drugs more frequently and 
with greater quantities consumed during college and after college compared to other ethnicities. 16 

While students of color (particularly those at historically black colleges and universities) consume 
alcohol and other drugs with less frequency and in lower quantities than their white counterparts, 17 
these patterns are mediated when students of color socialize in primarily white environments. 18 
Students of color in these environments also experience greater harms due to their alcohol use than 
white students at the same institution. 
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CHAPTER 1: Drug Misuse Prevention Landscape 


Figure 1 illustrates that college students and young 
people not enrolled in college use some drugs with 
the same frequency, such as consumption of flavored 
alcoholic beverages. Others are more prevalent among 
noncollege students, such as cannabis, cigarettes, or 
using hallucinogens. 

However, college students are more likely than 
noncollege students to have consumed alcohol in the last 
30 days and more likely to have been drunk. When asked 
about their past 30 days usage of specific drugs, college 
students reported consuming MDMA (i.e., Ecstasy), 
cocaine, and amphetamine as well as vaping (both 
cannabis and nicotine) at higher rates than noncollege 
students. 

SO WHAT DOES THIS MEAN? 

College seems to be a unique environment that 
depresses and exacerbates the use of specific drugs. 

We will go more in depth into why this happens when we 
discuss risk and protective factors in the next section, but for now, let’s look more closely at trends in drug usage 
among college students. While all campuses are different, the national trend data presented here provides a window 
into the changing norms and attitudes that drive drug use among college students around the country. (As a note, 
the following data are presented using “men” and “women,” a reflection of survey methodology. The important 
collection of gender nonbinary substance use data continues to grow.) 

ALCOHOL: Alcohol continues to be the most widely used drug on college campuses, with 75% of students reporting 
they had used alcohol in the past year. 19 Reflecting larger trends nationwide with the growing norm of women’s 
heavy use of alcohol, 40.3% of college women reported being drunk in the last month, compared to 35.5% of college 
men. Similar numbers of college-age men and women reported drinking flavored alcohol beverages in the past 
month (31.9% vs. 29.1%, respectively). 20 

CANNABIS: It likely comes as no surprise that after alcohol, cannabis is the most widely used drug on campus. 
Almost 25% of full-time college students reported using cannabis at least once in the last month, and 6% reported 
daily use (20 or more times in one month). While monthly use is similar for college-aged men and women, daily use 
is twice as high among men. 21 

CIGARETTES: Around 15% of college students reported using cigarettes or small cigars in the past year. 22 One in 
five (20.3%) of college-age men reported using cigarettes in the past year, compared to 12.2% of college women. 

VAPING: Vaping, or using electronic drug delivery systems for both cannabis and nicotine, continues to rise in 
popularity among young people. Among all college students, vaping cannabis is popular, with 20% reporting they 
had vaped within the past year, and 11% reporting they had vaped in the past month. 23 Matching nationwide trends 
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in the popularity of JUULand other e-cigarette delivery platforms, one in four college students reported vaping 
nicotine in the past year, with more men (33.7%) than women (22.1%) reporting use. 24 With the rising numbers of 
vaping-related lung illnesses, this popular method of drug delivery is ripe for prevention and education efforts. 25 

AMPHETAMINES: Nonprescription use of amphetamines used to treat ADHD remains steady for college students, 
with 11% reporting use of Adderall within the last year, and 1.3% reporting use of Ritalin. 26 Adderall use is higher 
among men than women, while Ritalin use rates are similar for both genders. 27 

COCAINE: One in 20 college students (5.3%) reported using cocaine in the past year. 28 College men reported higher 
annual cocaine usage (7.0%) than college women (4.3%). 

HALLUCINOGENS: Similar to cocaine, one in 20 college students reported past year usage of hallucinogens 

(5.2%). 29 Most commonly used were LSD (4.2%) and MDMA (4.4%). Rates for both are higher for college men than 

women. 30 

OPIOIDS (INCLUDING HEROIN): Heroin use, both snorted and injected, was virtually unreported by college 
students of both genders (less than 0.5%). Nonmedical use of opioid-based drugs were similarly low, with 1.4% 
reporting past year use of OxyContin, and 1.6% reporting past year use of Vicodin. 31 Rates for OxyContin use was 
almost equal among college-age men and women (1.9% v. 1.4%) and almost three times as high among men than 
women for Vicodin (2.3% v. 0.8%). 


The Impact of Environment 

Environment plays an indisputable role in protecting or promoting health behaviors. College campuses provide 
a unique combination of risk and protective factors with respect to drug misuse among students. We will revisit 
risk and protective factors in Chapter 3 when we work on needs assessment, but for now, we want to provide a 
framework for understanding how these factors influence college students. 
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Risk and protective factors exist in multiple contexts as illustrated in Figure 2. 
Figure 2. Socioecological model 32 



Each circle holds different factors that could either put a young adult at risk for or protect against later drug 
use. These factors do not stand alone; they are correlated and cumulative. They are also influential over time. 
Researchers refer to this cumulative effect as a developmental cascade, meaning that an individual’s collection of 
risk and protective factors can influence the likelihood of developing substance use disorders. 33 

For example, consider two hypothetical high school seniors from the same upper middle class neighborhood 
(community): 

1. A young man with friends who do not drink alcohol (peer) in a family where alcohol is moderately consumed 
(family), without a propensity toward risk-taking (individual) 

2. A young woman with friends who do drink and smoke cigarettes, in a family where alcohol use is moderately 
consumed (family), with poor impulse control (individual) 

Despite the similarities in their communities and in their family’s consumption of alcohol, the two young adults 
bring different risk and protective factors with them when they enter college. The young woman has both peer 
and individual risk factors that may influence her alcohol and other drug (AOD) use in college. Research suggests 
multiple risk factors for alcohol and drug misuse in college (Table 1). 
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Table 1. Research-supported risk factors for alcohol and drug misuse in college 


Risk Factor Description 

(Individual) 


Experiencing financial 
stress 

Students who reported experiencing financial stress within the last year were 
more likely to use nonprescription stimulants than students without financial 
stress. 34 

Experiencing stress in the 
past year 

Students who reported higher levels of life stress in the past year were more likely 
to use nonprescription stimulants than students who were not stressed. 35 

Adolescent history of 
depression 

Students who reported an adolescent history of depression had a greater 
tendency to misuse alcohol in college. 

Tendency toward 

sensation-seeking 

behaviors 

Students who reported a willingness to engage in sensation-seeking behaviors 
also reported lower perceived harm from alcohol and other drugs. They were also 
more likely to experiment with alcohol and drugs and to engage in higher rates of 
use. 36 

Current use of alcohol, 
tobacco, or cannabis 

Students who currently use alcohol, tobacco, or cannabis were more likely to 
initiate use of e-cigarettes (vaping) as a delivery method for nicotine. 37 


Risk Factor Description 

(Interpersonal) 


Family history of 
depression 

Students with a family history of depression were more likely to misuse alcohol in 
college. 38,39 

Perceived marijuana use 
by peers 

Students who perceived peers as engaging in heavy marijuana use (correctly or 
incorrectly) had a greater likelihood of developing a substance use disorder. 40 

Family environment 
favorable to drinking and 
smoking 

Students who reported growing up in families where alcohol and cigarette 
smoking were normalized had a higher likelihood of high-risk use of both 
substances in young adulthood. 41 

Perceived peer alcohol 
use in first year of college 

First-year college students who perceived high-risk drinking as a norm among 
their peers had a higher likelihood of engaging in heavy episodic drinking. 42 


Risk Factor Description 

(Community Level) 


Campus normalization of 
alcohol and drug use 

While in high school, students who are planning on attending college report 
lower rates of both alcohol and drug usage than their peers who are not planning 
on attending college. Within one year of arriving on campus, however, rates of 
alcohol and almost all other drug use (with the exception of opioids) are higher for 
college students compared to their noncollege peers. 43 
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Lack of parental 
supervision 

For most college students, matriculation into a university represents the first time 
they have lived away from home and the lack of adult supervision can lead to 
riskier behaviors for some students. 44 

Increased availability of 
alcohol and drugs 

Colleges are mixed-age environments where 25% of students are legally able to 
purchase alcohol and, in states where legal, cannabis as well. Combined with the 
lack of parental supervision, this ease of obtaining alcohol and drugs can drive 
high-risk use. 45 

Living off campus 

Research has shown that students who live off campus reported drinking at a 
higher frequency and being drunk with greater frequency than students who live 
on campus. 46 

Living in a residence 
hall suite (rather than an 

individual room) 

With more and more colleges designing apartment-like living for their on-campus 
students, it’s important to note that students living in these environments: 

» Are more likely to drink with greater frequency 

» Consume more alcohol when socializing 

» More frequently engage in heavy episodic drinking 

» More frequently drink in their residence halls 47 


While it may seem that the college environment is riddled with risk factors, most campuses also confer a wealth of 
protective factors on college students that can be easily enhanced (Table 2). 

Table 2. Research-supported protective factors against alcohol and drug misuse 


Protective Factor Description 

(Individual) 


Protective Factor 
(Individual) 

Description 

Negative attitude toward 
alcohol 

College students who held negative beliefs about alcohol and its effects were 
less likely to engage in high-risk drinking. 48 

Working for 10+ hours for 
salary 

Students who worked a paid job for more than 10 hours a week were less 
likely to engage in high-risk drinking. 49 

Abstaining in high school 

Students who abstained from alcohol in high school had a greater likelihood of 
abstaining from alcohol in college. 50 

Religious commitment and 
coping 

Students who reported that they used religion as a coping mechanism for 
stress and who participated in religious communities were less likely to misuse 
alcohol. 51 ’ 52 
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Protective Factor 
(Interpersonal) 

Description 

Parental monitoring 

College students who reported high levels of parent engagement in their lives 
during college were less likely to engage in heavy episodic drinking or initiate 
marijuana use. 53 

Perceived peer disapproval 
of alcohol and other drug use 

Students who reported abstaining from alcohol cited peer disapproval as one 
of the reasons they maintained abstinence. 54 

No family history of alcohol 
misuse 

Students with no history of alcohol misuse showed a decreased likelihood of 
having positive alcohol expectancies or engaging in high-risk alcohol use. 55 

Protective Factor 
(Community Level) 

Description 

Involvement in service-based 

extracurricular activities 

Students who reported involvement in service-based extracurricular activities 
such as volunteering reported lower levels of alcohol consumption. 56 

Alcohol-free events and 
programming 

Students who attended alcohol-free events and programming reported 
consuming less alcohol that day/night than students who did not attend such 
programming. 57 

Living in substance-free 
housing 

Students who either chose or were assigned to substance-free housing had 
lower rates of alcohol and other drug use when compared to students who 
were in traditional on-campus housing. 58 


When considering prevention planning, it’s important to do an inventory of your campus and to consider the unique 
risk and protective factors that the campus environment confers upon students. What factors on your campus 
promote alcohol and drug misuse? What factors protect against it? These are questions that will affect your work 
throughout this guide. 


Higher Risk Groups of Students 

In addition to considering community-level risk factors, 
it’s also important to understand the subgroups of 
students on your campus who may be at higher risk for 
substance misuse. Over the past 20 years, researchers 
have identified certain students who may be more likely 
to use or misuse alcohol and drugs. However, since every 
campus is different, it’s important to collect substance 
use data to be able to understand patterns of use by 
subpopulations on your campus. 
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CHAPTER 1: Drug Misuse Prevention Landscape 


ATHLETES Athletes are more likely than nonathletes to consume alcohol frequently and heavily. 59 These 
consumption patterns are consistent for athletes across the college spectrum. College athletes in the Division III 
system consume marijuana and amphetamines more than their Division I counterparts. 60 

FRATERNITY AND SORORITY STUDENTS: Research has consistently shown that students in the Fraternity & 
Sorority Life (FSL) system drink alcohol more and have more lax views on the harms of alcohol consumption. 6162 
Fraternity and sorority members also report consuming nonprescription stimulants and pain medications, cannabis, 
hallucinogens, and other drugs at higher rates than their non-FSL peers. 63 

LGBTQIA+ STUDENTS: Students who identify as LGBTQIA+ experience higher rates of depression, anxiety, and 
panic disorders, as well as higher rates of substance misuse. 64 Substance misuse is higher among female students 
with partners of both genders than among males with partners of both genders. 65 

STUDENTS WITH CERTAIN MENTAL HEALTH CONDITIONS: Numerous studies have found relationships 
between students with certain mental health conditions and high-risk alcohol and other drug use, with some 
scholars theorizing that these are attempts to self-medicate. Nevertheless, certain drugs are more frequently 
used by students with specific mental health conditions: depression and cannabis use, anxiety and cigarette use, 
panic disorders and sedatives. 66 ’ 67,68 While these are not one-to-one correlations, the relationships are of note in 
prevention planning. 


Table 3: How substance misuse affects college academic mission 


One of the best ways to sustain a prevention program’s efforts is to link those efforts to a 
larger set of issues that the college or university president cares about, such as student 
retention and success, student health, campus security, and fiscal management. Part of 
your job is to make it hard to remove a drug misuse prevention program, or parts of it, by 
promoting its overall value to the school. 

Health and safety 

Students who binge drink (more than five drinks at a time) are more likely to 
report injuries and engage in unplanned sex. Students at schools with high 
rates of binge drinking are more likely to be assaulted by a drunk student. 69 

Student retention 

Students who binge drink are more likely to experience early departure from 
college, and they have less favorable job prospects after graduation. 70 Students 
who use cannabis more than five times a year are more likely to experience 
problems with concentration and miss class than those who do not use. 71 

Campus safety 

Students who binge drink are 3.5 times more likely to experience violence 
than students who do not binge drink. Almost 20% of students also reported 
feeling unsafe due to another student’s drinking. 72 

Fiscal management 

A college’s finances are adversely affected by students’ misuse of alcohol and 
other drugs, including lost tuition from students who drop out, time spent by 
public safety officials and campus administrators on AOD issues, and legal 
settlements for student death or injury due to AOD misuse. 73 
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Federal Policies That Affect Prevention on College Campuses 

The final part of the college landscape that we must understand in order to do effective drug misuse prevention is a 
collection of federal policies that affect our work with young adults on campus. The three most relevant to substance 
misuse work on campus follow. 


Family Educational Rights and Privacy Act 

FERPA provides parents of children under the age of 18 rights to access their child’s educational records at any 
school that receives funding through the U.S. Department of Education. At the age of 18, however, FERPA rights 
transfer to the child who is now considered an “eligible student.” For all intents and purposes, this means the 
student now has control over what any party, including their parents, sees in their educational record. 74 

FERPA has implications for the types of interventions that can be done for college students struggling with alcohol 
and other drug use. For example, a student failing a course due in part to their drug or alcohol use is entitled to 
privacy regarding their academic records under FERPA. Rather than a college administrator reaching out to a parent 
for help, the student must initiate and consent to such contact. Understanding the autonomy that FERPA provides 
to students can be instrumental in the development of prevention interventions. 


Health Information Portability and Accountability Act 

Like FERPA, HIPAA affects the amount of information that a school can share about a college student’s health 
information with a parent. Once a child is 18 years old, HIPAA designates them as an adult, and their medical 
information may not be shared without their consent. HIPAA follows state medical privacy laws, which may vary in 
the amount of information that may be shared between adult children and their parents. 75 

However, some states have made changes to medical privacy protections under HIPAA in response to the opioid 
crisis. In particular, medical providers may be allowed to contact parents of adult children who do not have the 
capacity to consent to sharing their medical information due to an opioid or mental health emergency. Check your 
state’s medical privacy laws. 


Drug-Free Schools and Communities Act Amendments of 1989 

DFSCA “requires institutions to curb harmful and illegal substance use by distributing a comprehensive policy, 
enforcing alcohol and other drug-related standards of conduct, and implementing strong prevention programs.” 76 
Schools must also publish a biennial review of their alcohol and other drug policies and programs. Although 
guidelines have not been updated since 1989, any school that accepts federal funding must comply with DFSCA. 

In the past five years, studies have revealed that many colleges and universities are not compliant with DFSCA, 
with the most common noncompliance being the failure to produce a biennial report. 77 For prevention professionals 
on campus, the most important part of DFSCA relates to the requirement to create and evaluate a comprehensive 
substance misuse prevention program, enact policies to prevent alcohol and other drug misuse, and ensure that 
policy violations are sanctioned consistently. 
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Next Steps 

How are you feeling? We’ve gone over a lot of information! We now have a historical perspective on how our 
country’s leaders have thought about drug misuse prevention, research, and policy. We’ve considered how campus 
environments confer unique risk and protective factors on students’ drug use and on groups of students we should 
be thinking about. And we’ve reviewed some federal policies that will affect how we do effective drug misuse 
prevention. 

Now, let’s turn our attention to understanding the SPF and how it can be of use to us in our work. 
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CHAPTER 2 


Using the Strategic Prevention Framework to 
Prevent Drug Misuse Among College Students 


The Strategic Prevention Framework has lasted the test of time. It forces your 
program to be purposeful, strategic, and intentional. Working through the SPF 
helps you make decisions based on data and evidence-based practice. I may be 
doing something different than you are but that’s because my data is different. My 
prevention program won’t look exactly like yours and that’s okay. 

—Fran Harding, former Director of the Center for Substance Abuse Prevention at the Substance Abuse and Mental Health 
Services Administration (SAMHSA) 
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Imagine a night on your campus at the beginning of the academic year. The late summer heat still hangs in air, and 
students are not yet busy with homework and tests. Some students are having a party in a residence hall room, and 
at that party, a student overdoses and dies. 

It is not an understatement to say that this situation is a nightmare for college and university administrators. As 
health and wellness professionals, losing a student to overdose is our worst fear, compounded by the almost 
immediate slide into a difficult cascade of events. From the horror of telling parents their child has died to the 
immediate spotlight of negative media attention on your campus to addressing the confusion and distress of your 
student body after the loss of one of their own, a student death from drugs or alcohol affects the whole campus 
environment. 

Yet in the face of these actions, most best practice documents aimed at higher education professionals provide tips 
for how to navigate the immediate aftermath of the tragedy. They advise administrators and faculty on how to help 
students who may be traumatized by the death, whether or not to cancel classes, and how to formulate a media 
strategy that includes talking points for local and national outlets. 

For those of us involved in substance misuse prevention and health promotion, this narrow focus on the tragedy 
itself can feel jarring. After all, we know that most student deaths from drug and alcohol overdoses aren’t one-off 
events. We know the role that environment plays in behavior, and those of us who have been on campus for a 
while have some idea of where “problem” substance misuse behaviors occur with regularity, whether it’s a certain 
residence hall, an off-campus street, a fraternity and sorority residence, or a specific athletic team or social group. 
Most of us yearn to understand more about why and how this could have happened. 

However, in the aftermath of a tragedy like this, we are usually asked by administrators to do something to make 
sure it doesn’t happen again: buy an online drug education program, give a talk in a residence hall, or create an ad 
campaign about the dangers of a drug. 

And although we know there must be a more measured and intentional method to address the underlying alcohol or 
other drug issue on campus, it can also feel impossible to slow the momentum and reassure worried administrators 
and parents. This is where the SPF can be useful in shifting the conversation. 

Strategic Prevention Framework 

Developed by SAMHSA in 2004, the SPF is a five-step process that provides a method to design and deliver a 
culturally appropriate, effective, and sustainable prevention program. 
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Strategic Prevention Framework (SPF) 

Rather than jumping to a solution for a substance misuse issue, the SPF guides prevention practitioners 
through a process that answers the following questions: 


o 


Ill 

»> 

» 

£ 

What is the 

What do 1 have 

What should 1 do, 

How can 1 put my 

Is my plan 

problem? 

to work with? 

and how should 

1 do it? 

plan into action? 

succeeding? 


In answering these questions, prevention practitioners gain a deeper understanding of the complexities that 
underlie substance misuse issues, from individual risk factors to environments where misuse is more likely to occur. 
More importantly, the SPF provides a method to build support and foster common understanding in a community on 
the reasons why substance misuse issues are occurring and how best to address them while considering the unique 
characteristics of the community. 


Figure 3. Strategic Prevention Framework 

The five questions listed above relate to the 
five steps of the SPF, which will be detailed in 
individual chapters in the guide: 

STEP 1: Assessment. Identify local prevention 
needs based on data. Related question: What is 
the problem? 

STEP 2: Capacity. Build local resources and 
readiness to address prevention needs. Related 
question: What do I have to work with? 

STEP 3: Planning. Find out what works to 
address prevention needs and how to do it well. 
Related question: What should I do, and how 
should I do it? 



Evaluation 


Sustainability 1 
and Cultural 


STEP 4: Implementation. Deliver evidence-based interventions as needed. 
Related question: How can I put my plan into action? 


STEP 5: Evaluation. Examine the process and outcomes of interventions. 
Related question: Is my plan succeeding? 
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All of the steps are guided by two central principles— cultural competence and sustainability —which should be 
integrated into each step of the SPF. 


Cultural Competence 

Cultural competence describes the ability of an individual or organization to interact effectively with members of 
diverse population groups. At a college or university, this means understanding that specific student communities 
on your campus may have very different ways of thinking about and understanding a substance misuse issue. For 
example: 

» Consider the terms and phrases used by a student community when discussing substance misuse problems 
and related behaviors. 

» Look for prevention interventions that have been developed for and evaluated with an audience similar to your 
student population. 

» Develop case examples that reflect students’ life experiences to supplement an intervention that is already 
underway. 

» Conduct follow-up interviews with students to understand program evaluation findings. 

At every step of the SPF, remember to consider the campus culture as a whole, as well as the specific student 
communities within the campus, to ensure that diverse members on your campus actively participate in, feel 
comfortable with, and benefit from your prevention practices. 

One way to work toward cultural competence is to practice cultural humility , 78 or the active practice of dismantling 
the biases and beliefs that we, as individuals, bring to our work with students and student groups on our campuses. 
Practicing cultural humility also means taking a close look at the historical biases and belief systems that operate on 
our campuses and working to dismantle those systems as well. For example, when working with fraternity or sorority 
students, our interactions are influenced by our own experiences, whether positive, negative, or neutral, with these 
groups. They are also affected by the cultural history and perceived value of fraternity and sorority groups on our 
campus. Cultural humility refers to a process of both personal and institutional self-reflection and self-exploration to 
ensure that we are learning from others rather than assuming or ascribing beliefs or values to individuals or groups. 
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0 Check In: How I Let Go of Preconceptions About a Student 
Group in my Prevention Work 

As health and wellness professionals on college campuses, many of you are typically master’s or doctoral- 
level educated. Plainly stated: You have attended many years of school. After so much schooling, it’s likely 
that you have ideas or beliefs about certain groups of students on campus. 

To do effective prevention, however, you must start as close as you can with a blank slate and use hard data 
and objective evidence to drive your strategic planning. 

The following questions will help you start to think through your educational history and biases and allow you 
to work toward dismantling them: 

» How is the campus I work at different from my own undergraduate institution? Which type of 
institution is “better” or “worse”? How did I determine that? 

» How do I feel about students who join fraternities and sororities? What is the value of these groups 
on a college campus? How do I know that? What personal interactions have I had with students in 
fraternities and sororities? 

» How do I feel about student athletes? How are athletes at large Division I schools different from 
athletes at smaller Division III programs? Where did these beliefs come from? What personal 
interactions have I had with student athletes? 

» How do I feel about students who experiment with drugs? What types of students participate in that 
culture? How do I know that? How do my own experiences with drug use (or lack of drug use) affect 
how I see these students? 

Use these questions as a check-in when you find yourself drawing conclusions or making judgments about a 
student group or population. Work hard to take yourself and your beliefs out of your prevention planning. 


Sustainability 

Equally important is the concept of sustainability, or the process of building an adaptive and effective system that 
achieves and maintains desired long-term results. To break the cycle of one-off programs and campaigns on your 
campus, you must do the following: 

» Think about sustainability from the beginning. Build community support, show results, and secure 
continued funding for prevention efforts. 

» Identify diverse resources. Look for people, partnerships, and materials to support prevention in unexpected 
places. 

» Invest in capacity. Find ways to teach others how to assess needs, plan, and deliver interventions. 
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» Build ownership among stakeholders. Communicate and connect with people on your campus. The more 
you inform and involve people, the more likely they will be to help sustain prevention efforts. 

» Identify program champions. Find individuals committed to substance misuse prevention. These people will 
be your program champions. Understand that some people are more excited about prevention—and more 
influential on your campus—than others. 

» Track and tout outcomes. Use strong evaluation methods to help you determine, and communicate to 
others, which prevention efforts are worth sustaining. 


Characteristics of the SPF 

The SPF is a dynamic and iterative process that encourages practitioners to go forward and backward in the steps 
as a part of planning. For example, if an intervention that is ready to launch doesn’t have the support it needs from a 
key campus group, don’t be afraid to go back and build capacity and buy-in. 

The SPF is also data driven, which undergirds everything from understanding the scope of a substance misuse issue 
to selecting an evidence-based intervention that is appropriate for your student population. As those of us who have 
worked on college campuses can attest, different years mean different issues with substance misuse: One year, 
students may report increasing use of e-cigarettes, while the next year, nonmedical use of prescription stimulants 
may be on the rise. Prioritizing data at every step of your prevention planning is the only way to know the extent of 
your campus’s substance misuse issues and how to best address them. 

Finally, the SPF is a team-driven approach, which may come as a relief for the vast majority of you on campus who 
do substance misuse prevention on your own. Every step of the SPF benefits from and relies on participation from a 
diverse cross-section of your campus population. These players may change and move in and out of your prevention 
programming process as your campus’s needs change. 

The rest of this guide details each step of the SPF, complete with interactive worksheets and stories from other 
prevention practitioners on campus who have done similar work. 
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How to Assess Drug Misuse 
on Your Campus 


I cannot imagine any case where you have an entire [AOD] survey that is negative. 
There are things that the majority of students are doing well, so you need to point 
out the balance between the positive and negative that helps build the true story 
from the data. Burying the data or not collecting it at all doesn’t help anybody. You 
need data to move forward. 

—Dr. Peggy Glider, Coordinator for Evaluation and Research, Campus Health Service, University of Arizona 
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For those of you who work every day to protect the health and well-being of your campus’s students, highlighting 
the substance misuse by those students can often feel like yelling into a void. Part of this difficulty is built into the 
college environment. A college education prioritizes new experiences and asks students to challenge long-held 
beliefs, which can make it tempting to see drug experimentation or even regular drug use as exploratory rather than 
problematic. 

One way to overcome this resistance is through the use of data, which also has the added advantage of being 
incredibly valued on a typical college campus. Students are asked every day to substantiate their ideas with data 
and facts, and the professors and administrators whose support you need to move prevention programs forward 
respect and respond to the marshaling of empirical evidence. 

How, though, do you begin the process of assessment? How do you quantify and define problem behaviors ? Coming 
back to the question of cultural humility, how can you work to make sure your own beliefs around drug use and 
experimentation do not color how you view and interpret your students’ drug use? 



When you conduct a substance misuse assessment, you are 
actually engaging in the process of completing four separate 
but related assessments. Specifically, you need to do the 
following steps: 

1. Assess problems and related behaviors 

2. Assess risk and protective factors 

3. Assess capacity for prevention 

4. Share your assessment findings 
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To begin, let’s start by explaining the difference between problems and behaviors. 

» PROBLEMS refer to the negative effects or consequences of substance misuse, either directly (such as 
overdosing on a drug) or indirectly (such as being less likely to graduate in four years due to drug use). 

» BEHAVIORS (or consumption) are a measure of how people use or misuse a certain substance. Patterns 
of consumption refer to how specific groups of people use or misuse a substance. On a typical campus, we 
may find many different consumption patterns for different student subgroups. 

It is worth noting that substance misuse can lead to many different problems. For example, use of Ritalin as a study 
aid by first-year students can lead to increased anxiety, decreased academic success, and increased likelihood of 
dependence on the drug. 79 

To assess substance misuse and its related behaviors, you must answer four basic questions: 

1. WHAT substance misuse problems (e.g., overdoses, alcohol poisoning) and related behaviors (e.g., 
prescription drug misuse, underage drinking) are occurring on your campus? 

2. HOW often are these substance misuse problems and related behaviors occurring? Which ones are 
happening the most? 

3. WHERE are these substance misuse problems and related behaviors occurring (e.g., at home or in vacant 
lots, in small groups, or during big parties)? 

4. WHO is experiencing more of these substance misuse problems and related behaviors (e.g., men, women, 
fraternity and sorority students, athletes, members of certain cultural groups)? 
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To answer these questions, you must access the information you have on hand about your students’ substance 
use. In particular, you will need to follow these steps: 


» Take stock of existing data: Start by looking for national college or university data and college-specific 
data already collected by others. Several organizations and researchers collect nationally representative 
samples of college students’ substance use behaviors, including the American College Health 
Association’s National College Health Assessment and the Core Institute’s Alcohol and Other Drug Survey. 
Surveys on your own campus may include those conducted by the campus medical service, the fraternity 
and sorority life office, or student affairs. Many states also have statewide agencies and AOD coalitions 
that may be useful in providing either data for your population or instruments that can be modified for your 
data collection needs. 


» Look closely at your existing data: Examine the quality of the data that you’ve found, discard the 
data that are not useful, and create an inventory of the data you feel confident about including in your 
assessment. 


» Identify any data gaps: Examine your inventory of existing data and determine whether you are missing 
any information (e.g., about a particular problem, behavior, or population group). 


» 


Collect new data to fill those gaps: If you are missing information, determine which data collection 
method (e.g., surveys, focus groups, key informant interviews)—or combination of methods—represents 


the best way to obtain that information. [Tip Sheet: Data Collection Methods: Pros and Cons; = 


Tip Sheet: Strategies for Conducting Effective Focus Groups; 


Informant Interviews Tip Sheet: Potential Challenges To Obtaining Useful Data ] 


Tip Sheet: Tips for Conducting Key 


Once you have all of your assessment data, analyze it according to the following criteria to determine your 
community's priority substance use problem(s): 


» MAGNITUDE: Describes the prevalence of a specific substance misuse problem or related behavior. 
Which problem/behavior is most widespread in your community? 

» SEVERITY: Describes how large an impact a specific substance misuse problem or related behavior has 
on the people or the community. Which problem/behavior is most serious? 

» TREND: Describes how substance misuse patterns and related behaviors are changing over time within a 
community. Which problem/behavior is getting worse/better? 

» CHANGEABILITY: Describes how likely it is that a community will be able to modify the problem or 
related behavior. Which problem/behavior are you most likely to influence with your prevention efforts? 


Completing this part of the assessment will help you to identify the priority problem on your campus. 
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® Check In: What Happens If Our Needs Assessment Reveals High 
Rates of Alcohol or Drug Use? 

Let’s face it, collecting data on student alcohol and drug use can feel scary. Many of you probably have an 
impression of your campus’s substance use culture and anecdotal evidence about which students may be 
more likely to use. 

Substantiating all of those impressions with data can feel overwhelming: What happens when everyone 
knows exactly how much alcohol and drug use is happening on your campus? How can you even begin to 
change something so ingrained in your campus culture? 

Shifting your data perspective from fear-based to opportunity-based can take some practice, but it is 
invaluable for effective prevention planning. When interpreting your data, be sure to look for the good: 

» How many students are not using alcohol or drugs? 

» What are the reasons that students give for not using alcohol or drugs? 

» What story is your data telling you? When do students report their highest use? How can you put 
student alcohol and drug use into context with academic demands throughout the years? 

» Are there any drugs that the majority of your students do not use? What are they? 

» What types of protective strategies do you see being used by students who use alcohol or drugs? Are 
they going out with buddies? Do they eat before using? Do they intervene in risky situations? How can 
you build on protective strategies? 

Though the fear of having student alcohol and drug use data “out there” is real, don’t let it keep you from 
collecting the most accurate data you can. After all, you can only identify opportunities for prevention and 
areas where your students are doing well when your data is as robust as possible! 
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STEP 2: Assess Risk and 
Protective Factors 


Once you have determined your priority problem, you then need to understand the factors that make it more or 
less likely that your students will experience this problem. You do that by assessing risk and protective factors. 

» RISK FACTORS (e.g., low impulse control, peer substance misuse) are associated with a higher likelihood 
of developing a problem. 

» PROTECTIVE FACTORS (e.g., academic achievement, parental bonding, and family cohesion) are 
associated with a lower likelihood of developing a problem. 


In Chapter 1, we highlighted the different combination of risk and protective factors that influence college students’ 
drug and alcohol misuse. Understanding risk and protective factors is essential to prevention. Since you cannot 
change a substance use problem directly, you need to work through the underlying risk and protective factors that 
influence the problem. A prevention strategy or program can only make a difference if it’s a good match for both 
the problem and its underlying factors. [Tip Sheet: Protective Factors: Adolescence through Young Adulthood: j) 
Resource: Risk and Protective Factors associated with Binge or Heavy Episodic Drinking among Adolescents and 

Young Adults ] ^ 


Following are some key features of risk and 
protective factors: 

» Risk and protective factors exist in 
multiple contexts (e.g., individual, 
family, peer, and community). 

» Risk and protective factors are 
correlated and cumulative. 

» Risk and protective factors are 
influential overtime. 
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To understand this in more detail, let’s consider how risk and protective factors might affect the trajectory of two 
hypothetical college students, A and B (Table 4). 


Table 4: Risk and protective factors of two college students 


Key Features 

Risk (Student A) 

Protective (Student B) 

Multiple contexts 

» Student A entered college with a history 
of cannabis use. 

» Student B didn’t use cannabis in high 
school. 


» Parents used cannabis with her. 

» Parents didn’t use cannabis at home. 


» Peer groups in high school and college 
use cannabis. 

» Student B enjoys school and is excited 
about college, particularly about soccer 
and choir. 

Correlated and 

cumulative 

As Student A enters college, her 
regular cannabis use is causing her 
greater academic problems than she 
experienced in high school. To forget these 
problems and deal with the higher stress 
atmosphere of college, she uses cannabis 
more frequently and regularly. 

As Student B enters college, she belongs 
to several different peer groups: a club 
soccer team, an all-women’s choral group, 
and a cooking club. Though some in her 
various peer groups use cannabis, she 
never feels pressured to do so. She has 
tried it twice, and both times, she felt safe 
and cared for by her peers, but the next 
day, she didn’t like the physical effects on 
her body. 

Effect of a single 
factor 

The night before Student A has four 
final exams, she hangs out with her 
friends while they study. In an effort to 
concentrate, the group smokes cannabis 
while studying, and Student A passes out 
late at night. She misses her morning final 
exams and is groggy during her afternoon 
ones. She fails the semester and has to 

take a leave of absence. 

Student B keeps in touch with her parents 
and has an honest relationship with them. 
She shares her experience using cannabis 
with them, and she is surprised and happy 
to find out that her parents aren’t angry 
with her for trying the drug. Her parents 
share their own experiences with cannabis 
and have an open conversation with her 
about how drug and alcohol use can be fun 
but also how to recognize when things are 
getting out of hand. 

Influence over 

time 

Student A did not come back to college 
and graduate. The effect of her risk factors 
profoundly affected the trajectory of her 
life. 

Student B’s open relationship with her 
parents, her participation in numerous 
college activities, and her self-analytic 
behavior when experimenting with drugs 
and alcohol all work together to protect her 
and keep her healthy. 
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It is important to note that the underlying factors driving a substance use problem on one campus or among one 
student group may differ from the factors driving that same problem on a different campus or with a different 
student group. Effective prevention focuses on reducing the risk factors and strengthening the protective factors 
specific to the priority problem in your campus community and among your student groups. 


® Check In: How Cultural Perspectives on Mental Health Affect 
Substance Misuse Prevention on Campus - Voices from the Field 

Currently the associate director of Fraternity and Sorority Life in the Office of Student Engagement at the 
University of Denver (DU), Joe Espinoza has spent his career working on the front lines of student support. 

In his previous position as a case manager in the Office of Student Outreach and Support at DU, Espinoza’s 
one-on-one work with students facing mental health, academic, and substance misuse issues led to his 
belief in a systemwide approach to prevention. 

Espinoza highlights the disparities in access to mental health care and cultural stigma among students 
from different backgrounds as a potential area of intervention by prevention professionals. “We have some 
students who come to us with a history of mental health care. They have been working with a therapist since 
they were young and have a lot of family support for any issues that may arise, including substance misuse,” 
he explains. 

For other students, Espinoza notes that college is the first time they may feel able to access mental health 
care: “We also have students who feel like they need mental health or substance misuse services but want 
discretion and privacy so their families don’t learn about it.” Some of these students are willing to pay for 
mental health and substance misuse services out-of-pocket to avoid having insurance charges show up on 
family bills, explains Espinoza. 

Espinoza emphasizes the need for low-cost or free mental health and substance misuse services for all 
students. “Like many schools, we end up referring much of our one-on-one mental health and substance 
misuse counseling to off-campus providers due to high demand on campus,” says Espinoza. “We need to 
take into account that we have populations of students who may avoid accessing services if they can’t do it 
privately and inexpensively.” 

However, even with these challenges, Espinoza sees hope in how colleges and universities promote mental 
health and substance misuse services. “We have reached a point where we have reduced stigma around 
mental health so now students are talking more freely about it,” he explains. Increasing access for all, 
Espinoza believes, is a worthy next challenge. 
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CHAPTER 3: How to Assess Drug Misuse on Your Campus 


STEP 3: Assess Capacity for 
Prevention 


Now that you understand your priority substance misuse problem and the risk and protective factors that influence 
your problem, you must assess your capacity to engage in prevention. Capacity refers to two main components: 


1. RESOURCES refer to anything a community can use to help address prevention needs, such as the 
following: 

» People (e.g., staff, volunteers) 

» Specialized knowledge and skills (e.g., research expertise) 

» Community connections (e.g., access to population groups) 

» Concrete supplies (e.g., money, equipment) 

» Community awareness of prevention needs 
» Existing efforts to meet those needs 

2. READINESS is the degree to which a community is willing and prepared to address prevention needs. 
Factors that affect readiness include the following: 

» Knowledge of the substance use problem 

» Existing efforts to address the problem 

» Availability of local resources 

» Support of local leaders 

» Community attitudes toward the problem 


To assess readiness for prevention, it is often helpful to speak one-on-one with your campus’s decision-makers and 
student leaders. If individuals with access to critical prevention resources are not on board, then it will be important 
to find ways early on to increase their level of readiness. 

If your campus participates in institutional assessments, that data can provide a window into the resources that 
currently exist and highlight gaps (e.g., budget, staff) that you may need to address before undertaking a prevention 
effort. Reach out to your campus provost or office of institutional research for information on what types of 
assessment data you may be able to access. 

Finally, you can use one of many organizational assessment tools that have been developed in the nonprofit sector 
to assess different aspects of capacity. The Hewlett Foundation has a comprehensive database of assessment tools 
that may fit your capacity assessment needs. [Tool: Database of Organization Assessment Tools ] 
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Understanding local capacity, including resources and readiness for prevention, can help you do the following: 

» Make realistic decisions about which prevention needs your campus is prepared to address 

» Identify resources you are likely to need, but don’t currently have, to address identified prevention needs 

» Develop a clear plan for building and mobilizing capacity (SPF Step 2) to address identified prevention 
needs 


STEP 4: Share Your Assessment 
Findings 


After completing a thorough assessment of prevention needs and capacity, you must communicate the key 
findings to prevention stakeholders on your campus. To do this effectively, consider who will be interested in your 
assessment findings and what format will work best for each audience. The following are some key strategies for 
sharing assessment findings: 


» Develop a full report: Your campus leaders and some of your prevention partners (such as health and 
wellness, student affairs, and fraternity and sorority life staff, as well as student leaders) will want the 
whole story, and it’s good to have all of the details in one place. 

» Highlight key findings: Many prevention stakeholders will want to learn about your main assessment 
findings. Compile key findings in slide presentations and handouts that you can use for different 
audiences. 

» Customize your presentations: As needed, tailor your presentations or handouts by featuring the data 
that are most meaningful to each audience. This is particularly important when presenting assessment 
findings to key stakeholders, such as campus leaders, administrators, or student groups. If these 
individuals have specific questions or reservations, be sure to address them. 

» Solicit input from your campus community: In addition to sharing your findings, it is also important to 
find ways for your campus community to comment on those findings. They can help confirm that you’re on 
the right track with your prevention plans—or shed some light on confusing or surprising findings and help 
you get back on track. 


Finish Strong! 

Now that you’ve finished your needs assessment, you should understand the following: 

» The primary and secondary effects of drug misuse on your campus 

» The risk and protective factors that contribute to drug misuse for individual students and campus-wide 
» How to find data sources to assess the prevalence of drug misuse on your campus. 
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How to Build Capacity to Prevent Drug 
Misuse on Your Campus 


We put a lot of our time and energy in getting the reluctant on board. Instead, find 
your allies and your partners and start with them and build momentum from there. 
Those people will help you get the other ones onboard, and the reluctant wont 
matter anymore, because you’re building and growing your program. Shift the energy 
toward your allies and your partners. That’s where the momentum is. 


Diane Fedorchak, Interim Director of the Center for Health Promotion at the University of Massachusetts Amherst 









CHAPTER 4: How to Build Capacity to Prevent Drug Misuse on Your Campus 


If you’ve ever been at a higher education conference focused on alcohol and other drug misuse prevention, you 
likely will have seen a presentation that makes campus and community collaboration to address alcohol and other 
drug issues look easy. The presenter runs the university alcohol and other drug office and may describe working 
with law enforcement, local landlords, alcohol retailers, and bar and restaurant owners to address alcohol and other 
drug misuse by college students. All of the parties seem to be on the same page and committed to building a healthy 
community for their college-age residents. 

If you’re struggling to get your prevention program efforts off the ground, a presentation like this can feel frustrating 
and demoralizing. How do they do it? How are all their community partners committed to one goal rather than 
blaming one another for the problems caused by college students’ AOD consumption? 

The answer is the difference in the capacity of a community or a campus to take action to address prevention 
priorities (see Figure 4). 

Figure 4. Capacity formula (Capacity = Resources + Readiness) 


Capacity 


r k. A k 


RESOURCES KZ 3 READINESS 
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A community needs human resources, which includes people and volunteers as well as knowledge and skills, and 
structural resources, such as policies, laws, and funding. Readiness is a measure of the community’s willingness and 
motivation to address your identified prevention priority. 

Recall in the last chapter how we focused on finding your campus’s institutional assessment data to quantify 
these measures. After diving into these data, you may feel that you currently don’t have the capacity to move your 
prevention goals forward—but don’t worry! You can build capacity on your campus and in your local community to 
bolster support for prevention. 

Here are three tried and true methods to build capacity: 

1. Engage diverse community stakeholders 

2. Develop and strengthen a prevention team 

3. Raise community awareness of the issue 
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STEP 1: Engage Diverse 
Community Stakeholders 


Engaging a broad range of stakeholders is key to unlocking your campus’s capacity for prevention. Prevention 
practitioners on campus need diverse partners—from students to administrators to local community leaders to 
law enforcement—to share information and resources, raise awareness of critical substance use problems, build 
support for prevention, and ensure that prevention activities reach multiple populations in multiple settings with 
multiple strategies. 

The following are some of the different campus and community sectors you may want to involve in your prevention 
initiative: 


» Campus leaders 
» Higher education administrators 
» Student leaders 
» Student affairs staff 
» Fraternity and sorority life staff 
» Athletic coaches and staff 
» Campus health and wellness staff 
» Campus law enforcement 
» Local law enforcement 
» Local medical center staff 
» Local alcohol retailers 
» Bar and restaurant owners 
» Local prevention coalition members 
» Local residents 



Are there other partners you can think of? [Worksheet: Identifying New Partners * 

Worksheet: Analyzing Existing Partnerships ] ^ 

Once you’ve determined the stakeholders you currently have on board and those you’ll need to bring to the table, 
you next need to assess how much effort they can realistically put toward your prevention efforts. Your stakeholders 
will have varying levels of interest or availability to address your prevention efforts—which in no way negates their 
support for your effort. For example, some stakeholders may be willing to help out with specific tasks, while others 
may be willing to take on leadership roles. Having a clear understanding of how your stakeholders will work with you 
is a key step in building capacity. 
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The following are some different participation options (i.e., levels of involvement) for prevention stakeholders on 
your campus: 80 [Tip Sheet: Levels of Collaboration ] 


» NO INVOLVEMENT Stakeholders engage in separate activities, strategies, and policies. “You do your 
thing; we’ll do ours.” 

» NETWORKING: Stakeholders share what they are doing during interagency meetings; talk about campus 
and community issues in which they all have a stake; or communicate about existing programs, activities, 
or services. “Let’s talk and share information.” 

» COOPERATION: Stakeholders publicize one another’s programs in agency newsletters, write letters in 
support of one another’s grant applications, co-sponsor trainings or professional development activities, 
and/or exchange resources such as technology expertise or meeting space. “I’ll support your program, 
and you'll support mine.” 

» COORDINATION: Stakeholders serve together on event planning committees and community boards or 
implement programs and services together. “Let’s partner on an event.” 

» COLLABORATION: Stakeholders create formal agreements, such as memoranda of understanding 
or contracts; develop common data collection systems; partner on joint fundraising efforts; pool 
fiscal or human resources; and create common workforce training systems. “Let’s work together on a 
comprehensive plan to address the issue. After all, our missions overlap.” 


The next step is to find your new stakeholders by doing the following: 


» Call your contacts, particularly those with overlapping interests: Did you have a moment of connection 
with another campus staffer at an event about prevention issues? Have you reached out to other campus 
prevention professionals in your town or city? How about a local prevention coalition? Use all your 
connections, no matter how small, to grow your list of potential stakeholders to support your prevention 
efforts. 

» Attend and speak up at campus meetings and events: Armed with the data you collected from the 
assessment on how alcohol and other drug issues affect all aspects of student life, start attending and 
participating in campus meetings and events focused on academic success, student mental health and 
well-being, and community health. 

» Ask your partners to contact their partners: Don’t be shy about asking people you know to bring new 
and diverse partners to the table. If you have a specific student or campus leader you’d like to connect 
with, be explicit in your request. 

» Keep potential partners well informed about prevention activities and progress: Consider sending out 
regular (but short) updates on your prevention efforts to your growing roster of stakeholders. It’s a great 
way to keep partners that may not currently have the capacity to be active in your efforts apprised of what 
you’re doing and how they could potentially get involved later. 
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» Meet with key players, including campus and student leaders and local decision-makers: Ask for a 

15- or 30-minute meeting and come with a focused and targeted presentation on your alcohol and other 
drug data. Answer their questions openly and transparently, ask for their impressions, and get their advice 
on who would be a good fit to join your prevention efforts. 

» Anticipate and overcome roadblocks: On any campus, there will be naysayers and voices of doubt, 
which is why it is crucial to address the concerns of those who might oppose or hinder prevention efforts. 


Once you have your list of stakeholders, it’s time to move them toward action. This is often where capacity 
building stalls, as it’s easier for most of us to network, or connect to one another, rather than ask for cooperation, 
coordination, or full collaboration. Here are some ways to move your interested stakeholders to action: 

[Tip Sheet: You Gotta Hear This! Developing an Effective Elevator Pitch ] 


» Meet face-to-face to discuss overlapping goals and agendas: Get your 15- or 30-minute presentation 
ready and pound the pavement! 

» Extend an invitation to attend a prevention team or task force meeting: Keep your agendas tight and 
focused, and make sure you are only inviting stakeholders to a meeting when you have something specific 
you want to share or would like their help with. 

» Make more specific requests for involvement once prevention planning is underway: People love 
to be asked to help in ways that highlight their strengths. Ask your stakeholders to complete tasks that 
match their interests. 

» Extend invitations to attend future prevention events and activities: Even if someone can’t help now, 
make sure to keep inviting them as you never know when their schedule or interest in your work could 
change. 

» If nothing else happens, maintain the relationship by keeping stakeholders informed of prevention 
activities and progress: Even if you’re not at the stage to start an active collaborative effort, be sure to 
send out those short, regular updates on what you are doing to advance prevention on your campus. 
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Check In: How Do I Make Sure My Stakeholder Meetings Are 
Engaging and Building Momentum Toward Change? 


Once you’ve got stakeholders interested in meeting as a group or task force, you have to plan an agenda for 
the meeting. Planning a meeting that is interesting and drives your agenda forward is invaluable to effective 
prevention. 

Here are some tips to get the most of your stakeholders’ time: 

» Determine a purpose for the meeting, then set a date. You might want to discuss student alcohol and 
drug use data or do a deep dive into your campus’s alcohol and drug policies or get opinions from 
stakeholders on a key question. Whatever it is, make sure you know exactly why you want to bring 
people together before sending an invite. 

» Consider whom to invite. Not all of your stakeholders need to be at all meetings. After you determine 
your meeting’s purpose, look at your stakeholders and decide who really needs to be there. Share your 
meeting’s purpose with your invitees so your stakeholders can bring others who may be able to help 
meet the meeting’s goals. 

» Stick to a schedule. Break your agenda down to specific blocks of time (e.g., 5,10,15 minutes) and 
keep the meeting to one hour or less. Share your schedule before your meeting and provide copies to 
attendees at the meeting (or post it on a white board or screen). Do not go over the allotted time. 

» Stop meeting hijackers. Don’t let one person dominate with their impressions or grievances. Practice 
saying, “Thanks so much for that perspective. Let’s hear from someone else before we make a 
decision,” before the meeting so you are prepared to redirect a voluble attendee firmly and positively. 

» Follow up. Send a thank you and a list of meeting accomplishments, tasks delegated, and next steps 
within 24 hours after the meeting ends. 

Conducting great meetings takes planning and preparation—time that you may feel you could be using 
elsewhere. However, engaging meetings lead to passionate prevention task forces and are well worth 
the effort! 
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STEP 2: Develop and Strengthen a 
Prevention Team 


Full collaboration, the highest level of involvement, often takes the form of a prevention task force. While not all 
your stakeholders need to be involved at this level, your task force should include representatives from campus and 
community groups that are most vital to the success of your prevention initiative. Here are some ideas for building 
and/or strengthening your task force: [Tip Sheet: Beginning Your Collaboration: Tips for a Safe and Satisfying 
Journey ] 


» Identify and fill gaps: Once you have a team in place, ask yourself: Are your key campus and community 
groups represented? If you identify any gaps, try to fill them—but first make sure that your existing 
partners support additional recruitment. If current partners have reservations (e.g., “More people mean 
more opinions and conflict!”), take some time to point out, as specifically as possible, why you want to 
bring each new partner on board. 

» Build prevention knowledge: A truly representative task force means that members will bring diverse 
insights and experiences to the table, as well as varied knowledge and perspectives on the priority 
problem being addressed. Use a variety of strategies, including guest speakers and group trainings, to 
increase task force members’ understanding of the problem and effective prevention strategies. 

» Monitor and improve group structure and processes, as needed: Even the most well-informed group 
won’t be productive unless it functions well. To help your team work together effectively, discuss how you 
will share leadership, make decisions, divide tasks, resolve conflicts, and communicate with one another, 
as well as with the broader community. 
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STEP 3: Raise Community 
Awareness of the Issue 


By raising public awareness of your campus’s priority substance misuse problem, you can help garner valuable 
resources and increase campus readiness for prevention. 

The following are some strategies for raising awareness on your campus: 


» Meet one-on-one with public opinion leaders, such as student newspaper opinion columnists or student 
influencers on social media 

» Ask task force members to share information with their own groups 
» Submit articles to student and local newspapers 
» Share information on campus websites and social media outlets 
» Host campus-wide events to share information about and discuss the problem 
» Convene focus groups to get input on prevention plans 


It’s always helpful to think outside the box when looking for new ways to raise awareness on your campus. For 
example, your college may have a media studies department that can help you create a video about your campus’s 
priority problem and/or prevention efforts. You may have student social media mavens who are gifted at producing 
short videos or using photography to convey complexities. Which individuals and groups on your campus could help 
you reach out, spread the word, and get others involved? 


Finish Strong! 

Now that you’ve worked through how to build capacity for a prevention program, you should know the following: 

» Your campus’s stakeholders and which groups of people you need to connect with to ensure your prevention 
program will be successful 

» Names of champions on your campus who can promote your drug misuse prevention program and help you 
find allies for collaboration 

» Exactly how much work you need to do to spotlight issues of drug misuse on your campus 
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How to Plan a Successful Drug Misuse 
Prevention Program on Your Campus 


If you are developing prevention programs, I highly recommend working to 
understand initiatives and projects that are already doing this work across the 
country There’s a really strong community, and people who work in the prevention 
field want to share resources and share best practices to help one another. It’s really 
important to leverage that to your advantage. 


Dr. Erica Phillips, Associate Director in the Center for the Study of Student Life at the Ohio State University 






CHAPTER 5: How to Plan a Successful Drug Misuse Prevention Program on Your Campus 


For those of you who have spent time working to prevent alcohol and other drug misuse on college campuses, you 
know that there is a rotating cast of motivational speakers, online prevention programs, and in-person workshop 
facilitators that seem to make the rounds from one campus to another. One year, social norms experts are all 
the rage in your AOD professional group, and the next, everyone is wondering if the new online motivational 
interviewing-based drug misuse prevention program will work for their first-year student orientation programming. 

In many ways, we are fortunate to work among such committed and passionate prevention professionals, but 
having an excess of interesting and engaging prevention programs and workshops at our disposal can make the 
job of strategic prevention more difficult than it needs to be. How many of us have reached out for feedback to a 
professional group for a prevention program idea and ended up more uncertain than when we started? 

The SPF can help eliminate that confusion. It is grounded in the idea that every prevention plan is unique and should 
be designed to meet the specific needs of the community. For those of you who work on college campuses, this 
means doing a deep dive into the distinctive characteristics of your students and their alcohol and other drug usage 
patterns, and then crafting a prevention plan that is uniquely tailored to address their needs. 


In short, you need to do four things: 

1. Prioritize risk and protective factors 

2. Select appropriate interventions to address 
priority factors 

3. Determine how many interventions you can 
realistically implement 

4. Build a strategic plan (or logic model) and share 
with your stakeholders 



Prioritize Risk and Protective Factors 

Every substance use problem on every campus is associated with multiple risk and protective factors. Think about 
the wide combination of risk factors on your campus. Maybe you have large numbers of students living off campus 
or a high proportion of students affiliated with a fraternity, a sorority, or a student body who are likely to have used 
alcohol and drugs in high school. 

No campus AOD misuse prevention program can address all of these factors—at least not at once. So the first step 
in developing a prevention plan is to figure out which risk and protective factors to address first. To prioritize factors, 
it’s helpful to consider a factor’s importance and changeability (Figure 5). 
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Figure 5. Importance versus changeability 
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» IMPORTANCE: This describes how a specific risk or protective factor affects a problem. To determine a 
factor’s importance, ask yourself the following questions: 

• How much does this factor contribute to our priority problem? 

• Is this factor relevant, given the developmental stage of our focus population? 

• Is this factor associated with other behavioral health issues? 

» CHANGEABILITY: This describes a campus’s capacity to influence a specific risk or protective factor. To 
determine a factor’s changeability, ask yourself these questions: 

• Do we have the resources and readiness to address this factor? 

• Does a suitable intervention exist to address this factor? 

• Can we produce outcomes within a reasonable time frame? 

When developing a prevention plan, it is best to prioritize risk and protective factors that are high for both 
importance and changeability. A classic example for most college AOD misuse prevention professionals are 
intervention programs that are aimed at first-year students, either before they arrive on campus or when they first 
arrive. Numerous studies have shown that the first year of college is a crucial time for first-year students as they 
seek to understand and behave in ways that reflect the social norms of the campus (i.e., high in importance). 
Challenging norms around alcohol and other drug usage as soon as first-year students arrive on campus may 
prevent many of them from starting high-risk AOD use simply because they feel that it is expected of them (i.e., high 
in changeability). 
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If no factors are high for both, the next best option 
is to prioritize factors with high importance and low 
changeability. Since factors with high importance 
contribute significantly to priority substance misuse 
problems, addressing these factors is more likely to make 
a difference. And it’s easier to increase the changeability 
of a factor (e.g., by building capacity) than it is to 
increase its importance. 

However, in some cases your community may choose 
to address a factor with low importance and high 
changeability. Doing this can give your community a 
quick “win,” help raise awareness of and support for 
prevention, and increase the community’s capacity 
to address more important factors in the future. For 
example, a campus with a heavy partying and rambunctious off-campus student population that is embedded within 
a residential community might start by organizing a neighborhood cleanup on Sunday mornings for students and 
residents. Though the underlying partying issue is not being addressed, the cleanup builds capacity and fosters trust 
between students and residents, setting the stage for further interventions. 



Select Appropriate Interventions to Address Priority Factors 

Sometimes, prevention professionals may want to select interventions that are popular, that worked well on a 
different campus, or that they are familiar with. However, these are not great reasons for selecting an intervention. 

What’s more important is that the prevention intervention can effectively address the campus’s priority substance 
use problem and its associated risk and protective factors and that the intervention is a good fit for the campus 
community. 

Following are three important criteria for selecting appropriate prevention interventions: 

» Evidence based: Whenever possible, you should select evidence-based interventions (i.e., programs or 
practices that have peer-reviewed, rigorously evaluated empirical evidence of effectiveness). The best places 
to find evidence-based interventions are federal registries of model programs, such as NIAAA’s CollegeAlM. 
a compilation of evidence-based alcohol and other drug prevention programs on campus rated by efficacy, 
and SAMHSA’s Evidence-Based Practice Resource Center . Another excellent source of new and emerging 
interventions are evaluations published in peer-reviewed journals, such as the Journal of American College 
Health and the American Journal of Public Health. 

It’s important to note, however, that these sources are not exhaustive, and they may not include interventions 
appropriate for all problems and/or all populations. For college students, in particular, it can be difficult to 
find population-level studies of effectiveness, and it may be more useful to look for pilot studies that have 
promising results among small samples of college students who match your target population. 

[Tools: Preventing Substance Misuse among 18- to 25-Year Olds: Programs and Strategies: 

Tool: Preventing Youth Marijuana Use: Programs and Strategies ] ^ 
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Check-In: How Can You Determine The Strength of an Evidence- 
Based Substance Misuse Prevention Intervention? 


WHAT 

WORKS 

At least two experimental or quasi-experimental studies showing statistically 
significant results in the desired direction and the preponderance of all available 
evidence showing effectiveness. 

WHAT DOES 

NOT WORK 

At least two experimental or quasi-experimental studies showing statistically 
significant results showing ineffectiveness and the preponderance of all available 
evidence showing ineffectiveness. 

WHAT IS 

PROMISING 

At least one experimental or quasi-experimental study showing statistically 
significant results in the desired direction and the preponderance of the other 
studies showing effectiveness. 

WHAT IS 

UNKNOWN 

Any intervention that does not fall into one of the other categories. 


Source: Farrington, D. P., Gottfredson, D. C., Sherman, L. W., & Welsh, B. C. (2002). The Maryland scientific methods scale. In: L. W. Sherman, D. P. 
Farrington, B. C. Welsh, & D. L. MacKenzie (Eds.), Evidence-based crime prevention. London, UK and New York, NY: Routledge. 


» Conceptual fit: An intervention has good conceptual fit if it directly addresses one or more of the priority 
factors driving a specific substance use problem and has been shown to produce positive outcomes for 
members of the focus population. To determine the conceptual fit of an intervention, ask yourself, “Will this 
intervention have an impact on at least one of our campus’s priority risk and protective factors?" 

For example, screening and brief interventions, such as BASICS, are effective at challenging students’ 
beliefs about the prevalence of high-risk alcohol use on campus. If one of your risk factors is widespread 
misperception about heavy drinking, then BASICS may be a good fit conceptually. 

» Practical fit: An intervention has good practical fit if it is culturally relevant for the focus population; a 
campus has the capacity to support it; and it enhances or reinforces existing prevention activities. To 
determine the practical fit of an intervention, ask yourself, “Is this intervention appropriate for our campus?” 

Continuing with our BASICS example, to determine practical fit, you would need to assess whether BASICS 
works with your student population and, more importantly, if you have the capacity to support it. BASICS 
requires training for facilitators and dedicated time to do the intervention. You would also want to make sure 
that BASICS is targeting a unique need among your student population and not replicating other prevention 
efforts. 

Evidence-based interventions with both conceptual fit and practical fit will have the highest likelihood of producing 
positive prevention outcomes. 
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0 Check-In: Can I Use Evidence-based Interventions as Jumping Off 
Points for Innovative Prevention Programming?—Examples from 
the Field 

The field of college alcohol and drug misuse prevention research is robust and enthusiastic, but the fact 
remains that there are not a lot of population-level, evidence-based interventions to address substance 
misuse among college students. The strongest evidence supports brief interventions designed to promote 
individual behavior change. 

One such program is BASICS, or Brief Alcohol Screening and Intervention for College Students, a harm 
reduction program for college students who drink heavily. BASICS is aimed at students who drink heavily 
and also are at risk for alcohol-related consequences, both academic (e.g., failing classes) and personal 
(e.g., violence). The program uses a counselor trained in motivational interviewing who provides data on 
campus-wide drinking rates, challenges a student’s alcohol expectancies, and helps set new goals for 
alcohol use that are in line with the student’s stated life aims. 

Two researchers have taken the “basic” premise of BASICS and used it to develop innovative new 
programming: 

1. University of Tennessee Knoxville: Researcher Michael Mason’s team has been developing a mobile 
phone-based platform that uses text messaging, referred to as Peer-Network Counseling (PNC-text), 
to adapt the BASICS model for students who are heavy cannabis users. 81 Mason’s four-week pilot 
programs have been promising, showing that students are highly receptive to the text messaging 
format. Students completing the program report fewer heavy-cannabis-use days and relationship 
problems due to cannabis use after three months post-intervention. A larger multi-site study is now 
being conducted at Colorado State University and University of Tennessee Knoxville. 

2. University of Albany: Researcher Dolores Cimini’s team adapted the BASICS approach for student 
athletes, using aggregate athlete drinking data as the reference point. Her team tailored the 
intervention to challenge athletes to think about how their alcohol use affects their goals as athletes. 82 
After three months, athletes drank less, used more protective strategies, and experienced fewer 
negative consequences from their alcohol use. 

Bottom line? Don’t be afraid to look at evidence-based programming and see how you can adapt it for your 
student population. Reach out to the researchers who conducted the original studies and ask for help. Go 
forward and innovate! 
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Determine How Many Interventions You Can Realistically 
Implement 

In a comprehensive approach to prevention, interventions combine to have widespread reach, target multiple 
domains, and ensure cultural relevance. However, many campuses may not have the capacity to build such an 
approach. If a comprehensive approach is not a realistic possibility for your campus, you should instead focus on 
finding one intervention that will have the maximum impact. 

In short, consider the following with your list of possible interventions: 

1. Widespread reach: To produce population-level change, campuses should implement strategies with the 
greatest possible reach. To determine reach, ask yourself: 

• How many students will the intervention affect? 

• Which groups on campus will be affected by your efforts? 

While they can represent an important component of a comprehensive prevention plan, environmental change 
strategies—such as social marketing, campus policy development, and enforcement—have greater reach. No 
prevention plan is truly comprehensive without attention to environmental or contextual change. 

2. Multiple domains: According to the socioecological model, risk and protective factors operate at four levels, 
or domains: individual, family, school/campus, and community/town. A comprehensive prevention plan 
includes multiple interventions operating in multiple settings and across multiple domains. 
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3. Cultural relevance: Interventions must be responsive to, and appropriate for, the different cultural groups 
on your campus. This includes not only high-risk groups, but all students who will be part of the intervention. 
Throughout the SPF process, you must take steps to ensure the cultural relevance of your prevention efforts. 
Here are a few ways to ensure cultural relevance: 

• While conducting your assessment, map the cultural landscape to identify different student groups on 
campus. Make sure you know key student leaders in each group, and analyze assessment data by group. 

• While building capacity, share and discuss assessment findings throughout campus, invite student 
leaders and members of different student groups to participate in prevention planning, and make sure 
the planning team includes students with strong ties to groups at high risk. 

• While planning, recruit students from your target population to help identify appropriate interventions 
and convene focus groups with diverse students to obtain valuable feedback on potential interventions. 

Build a Strategic Plan (or Logic Model) and Share with Your 
Stakeholders 

A strategic plan or logic model is a graphic planning tool, much like a road map, that can help your team 
communicate where you want to go and how you intend to get there. A logic model includes the following 
components: a problem statement, risk and protective factors related to the problem, interventions to address these 
factors, and anticipated intervention outcomes (see Figure 6). 

Figure 6. A simple logic model template 


Problems and 
Related Behaviors 



Risk and 

Protective Factors 



Interventions 



Outcomes 


Outcomes are the changes that communities want their interventions to produce. 

Prevention outcomes fall into two categories: 

1. Short-term outcomes are the most immediate effects of an intervention. They have the following traits: 

• Are closely related to how well the intervention is implemented 

• Usually include changes in knowledge, attitudes, beliefs, and skills 

• Tend to be connected to changes in priority risk and protective factors 


Prevention with Purpose: A Strategic Planning Guide for Preventing Drug Misuse among College Students 


53 




















CHAPTER 5: How to Plan a Successful Drug Misuse Prevention Program on Your Campus 


2. Long-term outcomes are the ultimate effects of interventions after they have been in place for a while. They 
exhibit these traits: 

• Usually result from positive short-term outcomes that can, over time, lead to long-term behavioral 
changes 

• May take a long time to produce and measure—sometimes many years 

When developing a logic model, it’s important to work with an evaluator to identify a set of anticipated outcomes 
that are clear and measurable. Reach out to professors who teach research and evaluation methods for help. If they 
themselves are unable to help, they will likely know other evaluators who can help you determine your anticipated 
outcomes. [Worksheet: Developing Your Logic Model: Worksheet ] 0 


After completing your logic model for AOD misuse prevention on your campus, share it with these two 
important groups: 

» PREVENTION PARTNERS: This group includes the following: 

• The individual staff and students, student groups, and staff departments that participated in your 
assessment 

• People you brought on board during the capacity-building processes 

• People who will play a key role in your chosen prevention interventions. 

Note: Be sure that your logic model clearly communicates what your prevention partners hope to 
accomplish and how you will all work together to make it happen. 

» OTHER PREVENTION STAKEHOLDERS: This group includes funders as well as campus and community 
members and groups who may not be actively involved in prevention efforts (yet!). 


A logic model can help you build support for prevention overall, and mobilize the specific capacity needed to 
implement selected interventions. The more people who understand the problem and are on board with the 
prevention plan, the more likely it is that interventions will be sustained over time. 


Finish Strong! 

Now that you’ve planned your prevention program, you should have done the following: 

» Prioritized your campus’s risk and protective factors 

» Determined your long- and short-term outcomes for your prevention program using a logic model 

» Selected an evidence-based substance misuse prevention program that meets your campus’s needs and, if 
needed, adapted the program for your student population 

» Created a plan of action to implement your prevention program 
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CHAPTER 6 

How to Implement a Successful Drug Misuse 
Prevention Program on Your Campus 


We should help students to remember that their prevention professionals on campus 
are trained to know the spectrum of evidence-based strategies, but that students have 
a lot of freedom within those strategies. We want them to implement things that 
they can get excited about and get creative about and also get other students excited 
about, potentially having a healthier and safer and more productive and academically 
focused environment. 

—Joan Masters, Senior Coordinator of Partners in Prevention (PIP) 
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With your intervention plan and logic model now in hand, it can be tempting to jump right in and implement your 
prevention program with your target student population. Be honest with yourself: How many times have you found 
a great intervention program and done just that, maybe with an online intervention or a workshop for a specific 
student population? How did it work out? Have you ever wondered why an intervention wasn’t received as well as 
you’d hoped? Why didn’t the students you wanted to target show up or engage with your program? 

The success of a prevention intervention depends on careful planning during all of the SPF steps, including 
implementation. Taking your time during implementation is crucial to building support for your intervention and 
ensuring that your intervention reaches your target student population. 


There are three main tasks to consider during 
implementation: 

1. Connect with key implementation partners 

2. Balance intervention fidelity and adaptation: 

• Fidelity: Maintain core components 

• Adaptation: Modify with care 

3. Establish implementation supports 



Connect with Key Implementation Partners 

You’ve already identified and connected with key implementation partners while doing the assessment, capacity 
building, and intervention planning steps. These are the individual students, staff, and student groups who will be 
responsible for and/or involved in the delivery of your selected interventions. Sometimes these partners will want to 
make changes to the implementation plan. Even if they don’t, it’s important to communicate openly and make sure 
that all partners are on board with the implementation plan as you move forward. 

Consider a hypothetical intervention aimed at sorority students on a campus. The intervention consists of two 
60-minute group-based facilitated conversations that use the principles of motivational interviewing. During the 
program, the facilitator compares consumption data and beliefs about AOD use from one sorority to all the others on 
campus. The facilitator then guides the sorority into a discussion about values and how they align with the group’s 
AOD use. 

It is important for you to make sure sorority leaders on campus have bought into all aspects of the program, from the 
material and data being presented to the timeline and scheduling of the intervention. Ensuring that implementation 
partners and targeted student populations are invested in the program will set the stage for a successful 
intervention. 
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Balance Intervention Fidelity and Adaptation 

As you prepare to implement your selected prevention interventions, it is important to consider fidelity and 
adaptation: 


» FIDELITY is the degree to which an intervention, program or practice is implemented as intended. 

» ADAPTATION describes how much, and in what ways, an intervention, program or practice is changed to 
meet local circumstances. 


Evidence-based programs are defined as such because they consistently achieve positive outcomes. The greater 
your fidelity to the original intervention design, the more likely you are to reproduce these positive results. 

However, customizing an intervention to better reflect the attitudes, beliefs, experiences, and values of your 
focus population can increase its cultural relevance, even though such adaptations may compromise intervention 
effectiveness. Finding a balance is key to maintaining intervention efficacy. 

Let’s look closer at these two concepts. 


Fidelity: Maintain Core Components 


“Fidelity may be defined as the extent to which delivery of an intervention 
adheres to the protocol or program model originally developed.” 83 


Evidence-based interventions or programs are more likely to be effective when their core components, those 
elements responsible for producing positive outcomes, are maintained. Core components are like the key 
ingredients in a cookie recipe. You might be able to take out the chocolate chips, but if you take out the flour—a 
core component—the recipe won’t work! [Tool: What Are Core Components...and Why Do They Matter? ] 

So what are core components? Let’s go back to our hypothetical sorority intervention. After meeting with sorority 
leaders to get their buy-in for the intervention, the leaders request some changes to the program: 

» Changing the length of the intervention from two 60-minute sessions to two 30-minute sessions 
» Adding values-based content from their national office to the discussion of values. 

» Having the assistant dean of Fraternity and Sorority Life facilitate the program. 
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Which ones of these should you consider? In general, here are the guidelines for implementing an intervention with 
fidelity and maintaining core components: 

» Preserve the setting as well as the number and length of sessions. 

» Preserve key intervention content: It’s safer to add rather than subtract content. 

» Add new content with care: Consider intervention guidance and prevention research. 

» Identify the best possible candidate to deliver the intervention. 

So in working with the sorority leaders: 

» We would keep the intervention at two 60-minute sessions and explain why. 

» We could welcome the addition of values information. 

» We could ask if the Fraternity and Sorority Life dean might consider being trained in motivational interviewing 
principles and how to facilitate the intervention since they may be the best fit to work with this population. 


Adaptation: Modify with Care 

The degree to which an evidence-based prevention intervention is a good fit for the focus population is a prime 
consideration when selecting an intervention. However, as we’ve learned from our hypothetical sorority intervention, 
even when interventions are selected with great care, there may be ways to improve a program’s appropriateness for 
a unique focus population. 

Cultural adaptation refers to modifications that are tailored to the beliefs and practices of a particular group and 
enhance the cultural relevance of an intervention. To make an intervention more culturally appropriate, it is crucial 
to consider the language, values, attitudes, beliefs, and experiences of focus population members. 

When adapting an evidence-based intervention, it is important to consult with the following groups: 

» The intervention developers can provide information on how it has been adapted in the past, how well these 
adaptations have worked, and what core components should be retained to maintain effectiveness. 

» Members of your focus population can suggest ways to enhance the intervention materials to better reflect 
their concerns and experiences. Remember to practice cultural humility when receiving feedback. 

Keep in mind that adaptations can be planned to improve a program (as with cultural adaptation) or unplanned. 

It is important to be aware of the potential for unplanned changes that may occur during implementation, such as 
missed sessions if the campus is shut down due to bad weather, and to address any changes that might 
compromise intervention effectiveness (e.g., schedule make-up sessions so students don’t miss out on core 
intervention content). 
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Establish Implementation Supports 

Let’s return again to the question of why certain interventions succeed while others falter, even when you’ve taken 
steps to ensure that you choose interventions that are well suited to your populations and that address their risk or 
protective factors. What can you do to increase your chance of intervention success? 

As part of your implementation planning, you must consider the following: 84 

» Do you have a favorable prevention history with this student population? If you’ve had success 
implementing prevention interventions with this student population in the past, your students will likely 
be more ready, willing, and able to support the implementation of a new intervention. If your student 
population has had a negative experience with—or doesn’t fully understand the potential of—a prevention 
intervention, then it will be important to address these concerns early in the implementation process. 

» Do you have on-site leadership and administrative support? Prevention interventions assume many 
different forms and are implemented in many different settings. To be effective, interventions require the 
leadership of key student and staff groups and support from key stakeholders. 

» Did you choose the best practitioner to facilitate the intervention? When selecting the best candidate 
to deliver a prevention intervention, consider professional qualifications and experiences, practical skills, 
as well as fit with your focus population. Who is prepared to implement the intervention effectively? Who 
will make intervention participants feel comfortable? 

» Have you provided practitioner training and support? Pre- and in-service trainings can help 
practitioners responsible for implementing an intervention understand how and why the intervention 
works, practice new skills, and receive constructive feedback. Since most skills are learned on the job, it is 
also helpful to connect these practitioners with a coach who can provide ongoing support. 

» Have you developed a program evaluation plan? By closely monitoring and evaluating the delivery 
of an intervention, practitioners can make sure that it is being implemented as intended and improve it 
as needed. By assessing program outcomes, they can determine whether the intervention is working as 
intended and worthy of sustaining over time. (We will address this topic further in Chapter 7— How to 
Evaluate Your Drug Misuse Prevention Program.) 

» Do you have a clear action plan for implementation? Your plan should include (1) all implementation 
tasks, (2) deadlines, and (3) person(s) responsible. By working with implementation partners to develop 
this plan, practitioners can make sure that everyone is on the same page, and no key tasks fall through the 
cracks. 
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When you promote both fidelity and cultural relevance, and anticipate and support the many factors that influence 
implementation, you are ensuring that these efforts go a long way toward producing positive outcomes. But to 
sustain these outcomes over time, it is important to get others involved and invested in the prevention interventions. 
Find concrete and meaningful ways for people to get involved, keep cultural and public opinion leaders well 
informed, and get the word out to the broader community through media and other publicity efforts. 


Finish Strong! 

Before you move forward with implementation, you should know the following: 

» How your stakeholders will be included in your prevention program implementation plan 

» How you’ll balance fidelity (i.e., creating a consistent program) with adaptation (i.e., changing a program as 
needed) 

» How to keep program champions in the loop during prevention program planning and implementation 
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We must remember that prevention is really a long-term process. It will take 
ultimately sometimes eight to ten years to really see meaningful change in policy 
and commitment and behavior in our communities. Breaking things down into 
manageable steps, looking at data and assessing your outcomes along the way, can 
help you monitor your progress, so that you can actually keep track of it in real time. 
—Dr. Sally Linowski, Associate Dean of Students, University of Massachusetts Amherst 
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© Benefits of Evaluation 

Evaluation can help prevention professionals and communities accomplish the following: 

» Systematically document and describe prevention activities 
» Meet the diverse information needs of prevention stakeholders, including funders 
» Continuously improve prevention interventions 

» Demonstrate the impact of prevention interventions on substance misuse and related behavioral 
health problems 

» Identify which elements of a comprehensive prevention plan are working well 
» Build credibility and support for effective interventions in the community 

» Advance the field of prevention by increasing the knowledge base about what does—and does not— 
work 


How many of us enjoy prevention program evaluation? Working on a college campus where the focus is on furthering 
knowledge using fact-based evidence can make your work to change behaviors feel futile. How do you measure real 
behavior change? How do you account for shifts in AOD knowledge that don’t lead to behavior change? Why, despite 
all you are doing, do your students’ alcohol and drug use rates remain steady? 

When faced with such daunting questions, you might instead focus on measures of engagement: How many 
students attended your workshop, or how many students went through your screening and brief intervention 
program. 


However, a singular focus on engagement misses the real value of evaluation—rather than seeking to prove 
something, as many of the researchers on your campuses are working to do, evaluation seeks to improve processes. 
Understood in this way, evaluation is an exciting part of your prevention work as it can help you to enhance and 
tailor your programming to better fit your student populations. 


Evaluation is the fifth SPF step, and it involves 
examining both the process and outcomes of 
prevention interventions. When conducting an 
evaluation, you want to systematically collect and 
analyze information about prevention activities to 
reduce uncertainty, improve effectiveness, and 
make decisions. 


To better appreciate evaluation, let’s demystify 
some of its key components. In this section, we 
will highlight: 


» Different types of evaluation 
» Four basic evaluation principles 
» Evaluation tasks 
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Different Types of Evaluation 

There are two main types of evaluation: process and outcome. Process evaluation documents the implementation 
of a program or intervention. It can be used to improve an intervention’s delivery and enhance understanding of 
prevention outcomes. The following are examples of process evaluation questions: 

» To what extent were intervention sessions delivered as originally designed? 

» How many people participated in the intervention? 

» How many participants did not complete the intervention? 

» What, if any, adaptations were made to the intervention? 

This type of evaluation comes naturally to most of us who work with students. After all, measures of student 
engagement and interest are used on our campuses for everything from professor evaluations to justify funding for 
student programs or campus-wide initiatives. 

Focusing solely on process evaluation, however, limits our understanding of a program’s or intervention’s impact. 
Outcome evaluation, which measures the effects of a program or intervention following its implementation, can 
reveal whether the intervention produced the anticipated short- and long-term prevention outcomes and helped 
build support for those interventions that worked. The following are examples of outcome evaluation questions: 

» To what extent did students’ attitudes toward the priority problem(s) change? 

» To what extent did student rates of substance use behavior specific to the priority problem(s) change? 

Because behavior change is a slow and often nonlinear process for most people, this type of evaluation can feel 
daunting. We fear delayed results will confirm any negative perceptions about the substance misuse prevention 
work we are engaged in. Though it may feel easier to report on process measures, both types of evaluation are 
needed to produce interventions and programs that will have lasting impact on student behaviors. 

There are also two different ways for prevention staff and evaluators to work together: traditional and participatory. 

In a traditional approach to evaluation, an evaluator is hired to conduct an evaluation and works independently— 
interacting with your intervention and staff as needed to retrieve information. For a substance misuse prevention 
program, for example, you would provide the evaluator with your program’s materials, and the evaluator would 
define both the process measures and outcome measures and collect data to complete the evaluation. 

By contrast, in a participatory approach to evaluation, an evaluator is invited to take part in an evaluation as more 
of an advisor and a partner—interacting regularly with all involved as part of the group, rather than outside of it. The 
team, of which the evaluator is a member, works together to plan and carry out the evaluation. 
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A participatory method may feel more natural to those of you on campus as it draws on learning methods that your 
students already engage in. Participatory evaluation values the contributions of all who are involved with a program, 
from students to staff to campus leaders. In addition, a participatory approach can do the following: 

» Increase evaluation buy-in and evaluation capacity among participants 
» Increase the likelihood that the evaluation results will be valued and used 
» Increase the likelihood that the evaluation will be culturally appropriate and relevant 



Check In: What If My Outcome Evaluation Shows Our 
Intervention Doesn’t Change Student Rates of Substance Misuse? 


Outcome evaluation can feel like a day of reckoning. All of your work assessing data, building capacity, 
planning, and implementing leads to this: Does your program work as you intended? Does it affect student 
attitudes or beliefs or actual use of alcohol and drugs? 

It’s no surprise that process evaluation—measures of engagement, such as how many students showed up 
to a program or what they thought about the food—can feel easier to collect and report. 

Unlike process evaluation, outcome evaluation can’t be done well without bringing in an outside pair of 
eyes. And ironically, great outcome evaluation depends on great process evaluation. You can’t figure out 
why you didn’t achieve an outcome if you can’t evaluate every step in the process to determine where your 
intervention broke down. 

Here are some common reasons that your intervention may not have achieved its intended outcome: 

» Intervention needs to be done more frequently. 

» Intervention needs to be in a different format (e.g., online versus in person). 

» Intervention length is too long or too short. 

» Intervention components take longer to internalize than what the evaluation measured. 

» Intervention is not reaching the target audience. 

» Intervention isn’t culturally relevant for the target audience. 

» Intervention isn’t a good fit for the target audience. 

You’ll note that none of these have anything to do with your effectiveness as a prevention professional. Look 
at outcome evaluation as an opportunity to make your program stronger and more successful. Enlist an 
evaluation professional as part of your team, and evaluate with confidence! 
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Four Basic Evaluation Principles 

All evaluations—whether process or outcome, traditional or participatory—should adhere to the following four 
principles: utility, feasibility, propriety, and accuracy. 85 

To understand these principles in action, imagine that you are evaluating an intervention aimed at student athletes 
to address high-risk alcohol use during sports season. How would you make sure your evaluation is in line with the 
four principles? 

UTILITY is about making sure the evaluation meets the needs of prevention stakeholders, including funders. To 
increase the utility of the evaluation, you should: 

» Identify the evaluation needs of all key stakeholders 

(e.g., student athletes, coaches, alumni donors, campus leadership) 

» Make sure evaluators are trustworthy and competent 

(e.g., consider hiring evaluators who have experience in the world of college athletics) 

» Document findings so they are easily understood 

(e.g., keep technical jargon to a minimum and consider using the language and terms athletics and 
coaches use in your reports) 

» Share findings with stakeholders in a timely manner 

(e.g., create a plan to get evaluation results to all stakeholders before you begin) 


FEASIBILITY is about making sure the evaluation is realistic and doable. To ensure the feasibility of the evaluation, 
you should: 


» Establish data collection procedures that are practical and minimize disruption 

(e.g., consider the athletes’ schedule and time constraints when designing data collection) 

» Anticipate and address potential obstacles 

(e.g., opposition from campus leadership, alumni, or other interest groups) 

» Consider efficiency and cost-effectiveness 

(e.g., stay within your evaluation budget by using existing data) 
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PROPRIETY is about making sure the evaluation is conducted in accordance with legal and ethical guidelines and is 
consistent with each community’s cultural context. To support the propriety of the evaluation, you should: 


» Respect the rights and protect the well-being of all involved 

(e.g., how will you ensure student athletes’ privacy is maintained?) 

» Examine the intervention in a thorough and impartial manner 

(e.g., what are your biases or misperceptions about this program or these students, and how will you 
address them?) 

» Define how findings will be disclosed and who can access them 

(e.g., what types of reporting will you provide to your stakeholders? How will you allow access to data to 
ensure transparency and maintain privacy?) 


ACCURACY is about making sure the evaluation is conducted in a precise and dependable manner. To increase the 
accuracy of evaluation findings, you should do the following: 


» Clearly describe the intervention and evaluation procedures 

(e.g., use a logic model to depict different intervention components and who delivers each) 

» Gather and use information that is both valid and reliable 

(e.g., use standard measures or data collection tools that others have tested) 

» Systematically and appropriately analyze all information 

(e.g., start by looking at the quality of your data, including missing information and relationships between 
variables) 

» Justify and fairly report all conclusions 

(e.g., describe the limitations of your methods as well as the strengths) 


Evaluation Tasks 

Given all of the different methods of evaluation and principles to follow, the question of how exactly to begin an 
evaluation may feel increasingly opaque. However, for the past 20 years, the Centers for Disease Control and 
Prevention’s Framework for Program Evaluation has guided evaluators with a rigorous and clearly defined method to 
undertake public health evaluation. 86 

We have provided this framework in a checklist format. Remember that the best evaluations are collaborative 
processes that involve your stakeholders but also, crucially, engage the expertise of a professional evaluator. While 
there are some tasks that you will be able to complete on your own, don’t hesitate to reach out for guidance on the 
more technical aspects of evaluation design and methodology. 
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TASK 1. Engage Stakeholders 


0 


An evaluation stakeholder is anyone who cares about, or has something to gain or lose from, an intervention and its 
evaluation findings. 


Stakeholders include everyone who is: 

□ Involved in delivering the prevention interventions (e.g., intervention staff, student leaders, funders, 
community prevention partners, campus task force members) 

□ Served or affected by the prevention interventions (e.g., students, community advocacy and interest 
groups affected by the issue, campus leaders, public officials) 

□ In a position to do something with the evaluation findings (e.g., campus leaders, student leaders, 
prevention partners, campus task force members, funders, public officials, community members) 


© Why Engage Diverse Stakeholders? 

» Demonstrate respect for the many individuals and groups connected to prevention efforts 
» Obtain the help and support needed to conduct a thorough evaluation 
» Enhance understanding of evaluation among those involved in data collection and analysis 
» Ensure the cultural relevance and appropriateness of the evaluation design, tools, and findings 
» Increase the credibility of prevention interventions as well as the evaluation process and findings 
» Increase the likelihood that evaluation findings will be disseminated and used 
» Garner support for any efforts to expand and/or sustain prevention interventions 
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TASK 2. Describe the Initiative 


0 


Remember the logic model we created in Chapter 5: Planning that lays out exactly what your prevention initiative 
intends to do and achieve? This tool can help your prevention team communicate its plans to stakeholders and serve 
as a framework for evaluating the initiative. 


Specifically, your logic model has already identified the following: 

□ Priority substance use problem to be address by the prevention initiative 

□ Risk and protective factors, prioritized based on the degree to which they influence the problem at the 
local level and existing capacity to change them 

□ Evidence-based programs and strategies selected to address each priority factor 

□ Anticipated short- and long-term outcomes 


Recall that a process evaluation can be used to monitor and improve the implementation of your program or 
intervention, while an outcome evaluation can measure if and how your intervention is producing anticipated 
behavior outcomes. When a prevention initiative is laid out fully and clearly in a logic model form, it is much easier 
to identify appropriate evaluation questions and gather the data needed to answer them. [Tip Sheet: Using Process 
Evaluation to Monitor Program Implementation} | = | 
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TASK 3. Focus the Evaluation 
Design 


0 


Often, at the beginning of an evaluation, people jump right to thinking about how to collect data (e.g., “Let’s do 
a survey!”) before thinking through what data they’ll need. [Tool: Providing Evaluation Technical Assistance: 
Questions to Guide Evaluation Planning] > 


This task is a great place to involve an evaluation professional who can help you think through the 
following steps: 

□ Clarify your purpose: For example, do you want to find out if your interventions reached your focus 
population, or how well they worked to bring about change? Your purpose should be dictated by your 
stakeholders’ needs, including funding requirements, and guide all decisions that follow. 

□ Develop your questions: Once you’re clear on your purpose, you’ll need to develop evaluation questions 
that are specific to what you want to learn. Some questions can help you learn about the implementation 
of an intervention while others can help you learn about its outcomes. 

□ Select the right design: There are different ways to design, or structure, an evaluation. Some questions 
are best answered by gathering data from intervention participants and practitioners throughout 
implementation. Other questions are best answered by gathering data before and after an intervention, 
and/or from nonparticipants as well as participants. This latter approach allows for helpful comparisons 
and a better understanding of an intervention’s effects. [Tip Sheet: Selecting an Appropriate Evaluation 
Design] 

□ Choose appropriate methods: There are many different ways to gather the data you need. Which 
methods you select will depend on what you want to learn, your budget and timeline, and what’s most 
appropriate for your focus population. 

• Qualitative methods (e.g., interviews, focus groups) produce data that are usually expressed in 
words. They let you explore an issue or population in depth by answering questions such as Why or 
why not? and What does that mean? [Tip Sheet: Strategies for Conducting Effective Focus Groups 
Tip Sheet: Tips for Conducting Kev Informant Interviews] 

• Quantitative methods (e.g., surveys, checklists) produce data that are usually expressed in numbers. 
They allow you to draw general conclusions about an issue or population by answering questions 
such as How much? How many? and How often? 
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TASK 4. Gather Credible Evidence 


0 


How you gather data will determine how well you can answer your evaluation questions—and whether your findings 
will be taken seriously by others. 


This is another task where you may want to engage an evaluation professional to help you think through 
ways to increase the credibility of your evaluation by: 

□ Using quality tools and procedures: This means using data collection tools and procedures that are both 
valid and reliable. A valid tool measures what it’s supposed to measure. A reliable tool produces consistent 
results each time you use it. Selected tools and procedures should also be culturally appropriate. 

□ Taking a mixed-methods approach (i.e., a combination of quantitative and qualitative methods). This 
approach will allow you to examine your initiative from diverse perspectives, answer your evaluation 
questions more fully, and feel more confident in your findings. 

□ Providing training and support: Make sure that everyone involved in collecting and analyzing data gets 
the training and support they need to do it well. 

□ Gathering enough data Gather enough data from different sources to be able to draw conclusions with 
confidence—without going beyond your budget or missing important deadlines. Look back at the data 
you may have collected from your institutional assessment to determine capacity or from your needs 
assessment. How much of that data can you use now in your own evaluation? 

□ Managing the process: It’s important to take a systematic approach to storing and analyzing these data, 
as well as to developing and acting on your findings. 
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TASK 5. Justify Conclusions 


0 


Before you can justify your conclusions, you will need to analyze, synthesize, and interpret your evaluation data. 

□ Analyze: Analysis involves systematically examining each data source to determine key findings. 
Whenever possible, engage multiple reviewers in the data analysis process and make sure that everyone 
follows the same protocol. 

□ Synthesize: The next step is to compare and connect your results across data sources. By combining 
information from different data sources, you can detect areas of overlap and consistency—and identify 
new questions to explore when findings are inconsistent. 

□ Interpret: Finally, draw conclusions based on a careful examination of all your data. What positive or 
negative outcomes do your data reveal? Can you attribute these outcomes to the intervention or are other 
explanations possible? What decisions or actions do you recommend based on your conclusions? 


When analyzing, synthesizing, and interpreting evaluation data, it is important to involve the right people. These 
include individuals with research expertise, intervention staff, students, and other prevention stakeholders who can 
help increase the accuracy and cultural relevance of evaluation findings. 


TASK 6. Ensure Use and Share 
Lessons Learned 


0 


The best way to make sure that your evaluation findings will be used is to communicate them in ways that meet the 
needs of your diverse stakeholders. [Tip Sheet: Reporting Your Evaluation Results] [E| 

For each audience, ask yourself the following questions: 

□ What do they want to learn from the evaluation? Different audiences care about different things. For 
example, campus leaders will want to hear about the big picture. Are your interventions changing student 
behavior? Are you putting campus resources to good use? Because colleges and universities are invested 
in evidence and fact-based learning, you should also be prepared to engage in a discussion about your 
methodology by providing details of your evaluation procedures, methods, and findings. 

□ Which communication methods and channels are most appropriate? Consider how your different 
audiences get their information. You may be able to share information with some groups (e.g., campus 
departments, student groups) through meetings, campus newspapers, or by email. However, you may 
reach a wider group of students using student-driven posts on whatever social media is currently popular 
among your students. 
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© Ideas for Communicating Evaluation Findings 

» To share key evaluation findings with the public, submit a press release to local newspapers. 

» To get a large group on campus thinking and talking about evaluation findings, convene a campus 
town hall meeting. 

» To post on websites, distribute to mailing lists, and hand out at events, create fact sheets and/or 
infographics of key findings 

» To provide funders with a complete overview of the evaluation process and findings, write a full report. 

» To explore findings and potential next steps with student groups, schedule a small group presentation 
for each group. 

» To contribute to the prevention field, share your findings at a college health-focused conference or 
write and submit a journal article. 


Finish Strong! 

When planning a program evaluation, you should know the following: 

» The difference between process evaluation and outcome evaluation 
» How to determine which evaluation instruments can be adapted to assess your program 
» How to share evaluation results with your stakeholders 

» How you will celebrate your team and publicize your program’s success to the campus community 
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CHAPTER 8 


Advice for Established and Emerging College 
AOD Misuse Prevention Professionals: A 


Conversation with Dolores Cimini, 



University at Albany 


“Our goal is not necessarily to chase the drug when we are developing interventions 
but to see what the bigger issues are, the environmental issues and hold on to what 
the best practices are at the individual, campus and policy level.” 

—Dolores Cimini, Director, Center for Behavioral Health Promotion and Applied Research, University at Albany 






Chapter 8: Advice for Established and Emerging College AOD Misuse Prevention Professionals 


In this final chapter of the guide, we provide an inside look at the lessons learned by a prevention professional 
with a history of addressing AOD misuse among college students. We provide advice for both established and new 
professionals. 

For over 30 years, Dolores Cimini has been a mentor in the field of college AOD misuse prevention. Based at the 
University at Albany, one of the university centers of the 64-campus State University of New York system, Cimini has 
spent her career working directly with students. She has produced numerous peer-reviewed public health studies 
and is the co-editor of Promoting Behavioral Health and Reducing Risk Among College Students: A Comprehensive 
Approach (2018). Cimini currently runs the award-winning Middle Earth Peer Assistance Program at the University 
at Albany and is the director of the Center for Behavioral Health Promotion and Applied Research. 

Cimini is passionate about teaching and educating emerging professionals, and she is a well-loved educator in the 
School of Education at the University at Albany. She is also excited about the changes in the field of prevention: “We 

have steadily moved beyond traditional counseling services toward early intervention and universal intervention 
that reflect a true public health approach and engages the entire campus.” 

Cimini’s hard-earned advice is offered below. 


For Established Professionals 

Dealing With Changes in Upper Administration 

In her long tenure at the University at Albany, Cimini has worked under 13 university presidents. AOD issues are a 
charged issue on campus, and the fear of an unsupportive upper administrator is shared by many who do prevention 
work in these spaces. 



Cimini offers advice for weathering changes 
in transition: 


“When new presidents come in, by and 
large, they have a lot on their plate. 
They are learning about a new campus, 
meeting new people—their time is at a 
premium.” 


In response, Cimini and her team introduce themselves while remaining in the background. “We want new 
administrators to know that they have a program on their campus that is running well and moving forward,” she 
says. “We let them know that we’d love to talk to them about our program, but we know they have a lot on their plate 
right now. When we do that, we’ve found that they are not as concerned, and they let us do our work.” 


Prevention with Purpose: A Strategic Planning Guide for Preventing Drug Misuse among College Students 


74 





Chapter 8: Advice for Established and Emerging College AOD Misuse Prevention Professionals 


Cimini has found that using this approach establishes the competency of her office up front and also provides 
concrete data and program information for the president when they do have the time to meet with her and her team. 
Says Cimini, “When new administrators come visit, we provide a more comprehensive, data-driven picture of what 
we are doing. We also make it clear to them what support from their office looks like so they aren’t guessing about 
how they can help us or inform our work." 


Diversify Funding and Share Ownership to Embed Prevention into 
the Lifeblood of the Campus 

For professionals who have worked to establish a successful AOD misuse prevention program on their campus, 
Cimini offers methods for integrating programming into the day-to-day functioning of campus: “We look for ways to 
engage the whole campus: This is not the job of one office.” 


Cimini points to two areas for established professionals to pursue to ensure their prevention work remains 
central to the campus: 

1. Diversification of resources: “As part of their budget, many new prevention professionals may get some 
funding to implement strategies or programs. It’s important not to just rely on that one funding source,” 
Cimini explains. “If one is working in a grant-funded program, it’s important to not depend on that. 

Grants come and go, and budgets can be higher or lower depending on the particular academic year or 
institution. It’s important to look for other sources of not only funding support but also looking at how to 
sustain funding you do have.” One program that has benefited from this approach is the Middle Earth Peer 
Assistance Program, which is supported by a wide range of campus partners, including Student Affairs, 
Academic Departments and Student Government. 

2. Connection to academics: Another method Cimini uses at the University at Albany is to foster links 
between prevention programming and academics. She points again to the 50-year lifespan of the Middle 
Earth Peer Assistance Program as an example: “We’ve linked ourselves with the School of Education 
and are able to offer 3 credit hours each semester to students who participate in the Middle Earth 
Program. It’s wonderful because the students benefit, the university benefits, and our program benefits.” 

In the 2019-2020 academic year, Middle Earth had 157 student peer assistants and peer educators, a 
testament to the value of the program for all on campus. 
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Using Data to Stay Abreast of Emerging Drug Issues 

From cocaine in the 1980s to the rise of ecstasy and other club drugs in the 1990s to the misuse of prescription 
medications as study aids starting in the early 2000s, Cimini has seen a lot of trends in drug use over the course 
of her 30 years at the University at Albany. Throughout it all, she says, the popularity of cannabis has remained 
unchanged: “College students tend to believe that cannabis isn’t harmful and that perception has remained 
constant overtime.” 

So how does Cimini handle changes in drug popularity at the University at Albany? “We value data and collection 
of data. We also rely on receiving valuable information from our peer leaders since they work directly with the 
students,” Cimini says. “They serve as our eyes and ears for what’s happening on campus.” 

Cimini also stresses the importance of data analysis as a key part of the process, explaining that she has seen many 
colleagues collect data but then have challenges with finding the resources and expertise for data analysis. She 
acknowledges that she is lucky: “We have a graduate program with students and faculty who are interested in this 
area.” 

For those who are struggling to find help with data analysis, Cimini suggests, “Partner with faculty on your campus 
who may be interested in data collection and analysis, even if it’s a slightly different field. The skills are transferable, 
and it’s a win-win for everyone since you’re all working to build a healthier campus.” 


Working toward Holistic Prevention 

Cimini is sympathetic and attuned to the many challenges facing college students today: 

We can’t deny that college students are coming to campus with much more complex substance use and 
co-occurring mental health challenges. As a result, what we are seeing is an increased number of students 
who are facing potentially dropping out of school, stopping out of school, not graduating, or not moving into 
the workforce as has historically been the case. In addition to that, college students, particularly those at 
many public universities, are facing challenges such as financial concerns, food insecurity, or not being able 
to afford professional clothing when they do get job interviews. At times students need to decide between 
going to classes and doing their classwork as a top priority or needing to work and hold on to some role in 
supporting their families. 

As research has shown, financial stressors and mental health conditions are risk factors for substance misuse. 

With that in mind, Cimini believes in fostering partnerships and developing a comprehensive holistic approach 
to prevention. She explains, “Our goal is not necessarily to chase the drug when we are developing interventions 
but to see what the bigger issues are, the environmental issues, and hold on to what the best practices are at the 
individual, campus, and policy levels.” 

Cimini currently works with departments across campus to find ways to reduce the impact of risk factors, from 
establishing supports for first-generation college students to supporting the university’s growing initiatives around 
mindfulness and well-being programming aimed at reducing student stress and anxiety on campus. 
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Innovate Using Evidence-Based Programs as a Framework 

For established prevention professionals, the list of evidence-based AOD misuse prevention programs is well known. 
Conducting screening and brief intervention programs, establishing alcohol-free spaces on campus, and advocating 
for increased enforcement of AOD policies are the backbone of campus prevention programming. For many 
prevention professionals, implementing evidence-based prevention programs and policies comprises the majority of 
their efforts for their first 5 to 10 years on campus. 

Once those key evidence-based programs have been established, however, Cimini encourages professionals to 
innovate using the principles central to the success of evidence-based prevention programming. For example, her 
office received a federal grant to work with fraternity and sorority students on establishing a BASICS-like screening 
and brief intervention program. Fraternity and sorority leaders involved in the program’s creation advocated for the 
program to highlight fraternity and sorority values around shared identity, brotherhood/sisterhood, and campus 
reputation, in addition to providing individual and aggregate alcohol use feedback and alcohol expectancy data for 
each fraternity or sorority compared to all fraternities and sororities. 

While the program looks like BASICS and uses motivational interviewing principles in its approach, it’s entirely 
designed to fit the needs of the population. “We have to be willing to adapt our interventions, while keeping fidelity 
in mind, to meet our target population’s needs and to be responsive to their cultures,” Cimini explains. 


For New Professionals 

Understand Your Campus’s History around AOD Issues 

Taking the time to dive into your new campus’s past efforts at addressing AOD issues is well worth the effort, says 
Cimini. “You are walking into a living history,” she explains. “To conduct effective prevention, you must understand 
how your campus has worked with these issues in the past. What types of programs have they tried? How did the 
campus respond? How much support has the person working on these issues received in the past? Why is that? 
What resources has the program had in order to operate?” 

New professionals can gain a valuable perspective on the history of AOD misuse prevention on their campuses by 
using archival data, such as student newspapers, to learn how AOD use has been reported over time, along with 
conducting interviews with long-time campus leaders. 

In addition, Cimini advocates establishing a linkage with your primary supervisor and other campus leaders: “It’s 
really important to work with your supervisor or director to get the history and guidance of where your challenges 
may be and brainstorm how to address them. How did the program that I’m going to run in the next few years get to 
where it is? And what can I do to contribute to it in a unique way?” 
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Go on a Listening Tour 

Along similar lines, Cimini recommends taking a semester to conduct what she calls “listening tours” with your 
likely stakeholders. She recommends not only talking to faculty leaders and student life department heads but also 
interviewing student leaders and conducting focus groups with students who are traditionally considered “high risk” 
based on research. Learning how these groups of students have traditionally viewed the work of the AOD misuse 
prevention office is imperative before embarking on new prevention programming. Just as important is learning how 
the office has traditionally worked with faculty and other campus departments. 


For example: 

» How do faculty feel about the work you’re doing? 

» Are there clear lines of communication between faculty and the AOD office? 

» How have other student life departments worked with the AOD misuse prevention office in the past? 

» What types of initiatives do student life departments and faculty want for students around AOD issues? 


Don’t Rush into Programming—Take Your Time 

As a professional who has mentored generations of prevention professionals, Cimini understands the zeal of newly 
minted professionals to get started with the important work of crafting a prevention program. However, she cautions 
against jumping right in without doing a comprehensive needs assessment: “It can be tempting to start right in 
with prevention programming, but it’s important for us to really understand our stakeholders and target population 
groups, understand their cultures and their concerns, and be open and responsive to that.” 

For new professionals who may be worried that they might be viewed as ineffective if they don’t rush into enacting 
programs, Cimini recommends keeping key stakeholders engaged in your needs assessment and strategic planning 
processes. She says, “It’s really important to have a mind-set of collaboration with any stakeholder, not only how to 
collaborate with them, but to assess the strengths that they will be bringing to your work. Ideally, you want to place 
them in a position that will capitalize on their strengths. That may take some time to figure out. Don’t feel rushed.” 


A Final Note 

Like many others who have been in the field for years, Cimini is continually impressed by the quality and passion 
of new prevention professionals, saying, “We have come so far in how we understand these issues on campus, and 
there is a great deal of talent coming into these positions. It’s a truly exciting time for AOD prevention on campus!” 
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APPENDIX A: Additional Resources 


These resources are offered to inform your progress through the steps of the Strategic Prevention Framework. The 
inclusion of resources in this guide does not constitute a direct or indirect endorsement by DEA of any entity’s 
products, services, or policies, and any reference to an entity’s products, services, or policies should not be 
construed as such. 


Understanding the Problem 

Campus Drug Prevention (Drug Enforcement Administration) 

https://www.campusdrugprevention.gov 

The website was created for professionals working to prevent drug abuse among college students, including 
educators, student health centers, and student affairs personnel. In addition, it serves as a useful tool for college 
students, parents, and others involved in campus communities. The website offers valuable information, including 
data, news updates, drug scheduling and penalties, publications, research, national and statewide conferences and 
events, state and local prevention contacts, and resources available from DEA’s federal partners. 

Report to Congress on the Prevention and Reduction of Underage Drinking (2018) 

https://www.stopalcoholabuse.gov/resources/reporttocongress/RTC2018.aspx 

Compiled by the Interagency Coordinating Committee on the Prevention of Underage Drinking, this report provides 
policy summaries and state summaries identifying current legislative and other ongoing efforts. 

Facts on College Student Drinking 

https://www.stopalcoholabuse.gov/media/THMs/tipsresources/5486 UADPEI College Drinking Fact Sheet 

FINAL 4-2016.pdf 

This two-page fact sheet created by the Interagency Coordinating Committee on the Prevention of Underage 
Drinking provides an overview of the issue and breaks down binge and heavy drinking by gender, alcohol use 
consequences, and alcohol use prevention. 


Step 1: Needs Assessment 

National College Health Assessment (American College Health Association) 

https://www.acha.org/NCHA 

The National College Health Assessment offers national searchable survey results from 2015 until the present, 
reports and statistics, and access to published research. 

Monitoring the Future (National Institute of Drug Abuse) 

https://www.drugabuse.gov/related-topics/college-age-voung-adults 

Monitoring the Future provides the most recent data on substance use among this age group, including patterns 
of marijuana use, nonmedical use of prescription drugs, cocaine, and newer trends, such as synthetic drugs, 
e-cigarettes, and hookah use. It also provides other links of interest to educators, residence hall staff, counselors, 
clinicians, researchers who work with this age group, as well as students and parents. 
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College Drinking: Changing the Culture 

https://www.collegedrinkingprevention.gov 

A comprehensive resource from the National Institute on Alcohol Abuse and Alcoholism, this site is a central 
location for information related to alcohol use by college students, including the following: 

» College Alcohol Policies is a compilation of alcohol and other drug policies from thousands of colleges and 
universities across the United States. 

» College Alcohol Statistics provides updated national data on prevalence and consequences of alcohol use 
among college students. 


Step 2: Building Capacity 

Community Readiness Model 

https://tec.colostate.edu/communitv-readiness-2/ 

The Community Readiness Model was developed at the Tri-Ethnic Center for Prevention Research at the University 
of Colorado to provide communities with an easy-to-use method to assess resources and readiness to address a 
public health issue. 

Prevention Collaboration in Action Toolkit 

https://pscollaboration.edc.org 

Created by Prevention Solutions at the Educational Development Center, this toolkit offers tools and stories from the 
field on building partnerships and developing collaborations to reduce substance misuse. 


Step 3: Planning 

College AIM 

https://www.collegedrinkingprevention.gov/CollegeAIM/Default.aspx 

Developed by the National Institute on Alcohol Abuse and Alcoholism, College AIM is a toolkit designed to help 
schools identify effective alcohol interventions to address harmful and underage student drinking. 

Safer Campuses and Communities 

https://prev.org/SAFER/index.html 

Based on an NIAAA-funded study conducted at the University of California and California State University systems, 
SCC examined a variety of environmental-level strategies that could be implemented on campuses and in their 
surrounding communities. The site provides a free toolkit for fostering campus and community collaboration and 
implementing evidence-based environmental interventions. 

Evidence-Based Practices Resource Center 

https://www.samhsa.gov/ebp-resource-center 

The Substance Abuse and Mental Health Services Administration provides analyses, costs, and contact information 
for several individual- and environmental-level strategies to reduce alcohol use by college students. 
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Logic Model Development Guide 

https://www.wkkf.org/resource-directorv/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide 

Developed by W.K. Kellogg Foundation, this guide provides practical assistance to nonprofits engaged in program 
development, implementation, and evaluation processes. 


Step 4: Implementation 

National Center on Safe Supportive Learning Environments 

https://safesupportivelearning.ed.gov/events-products-and-ta/center-products-tools/higher-education-products 

This website provides evidence-based approaches to address alcohol and other drugs and issues of violence on 
campus. The site offers a variety of products: webinars and in-person learning opportunities, data resources, lessons 
learned profiles, and case studies from prevention professionals at colleges and universities. 

College Drinking: Prevention Perspectives - Lessons Learned at Frostburg State University 

https://store.samhsa.gov/products/College-Drinking/AII-New-Products/PEP18-FROSTBURG 

Per the website description, “This video shows the actions taken by Frostburg State University to reduce campus 
underage and harmful drinking.” 


Step 5: Evaluation 

An Overview of Quantitative and Qualitative Data Collection Methods 

https://www.nsf.gov/pubs/2002/nsf02057/nsf02057 4.pdf 

Created by the National Science Foundation, this guide provides information on quantitative and qualitative data 
collection methods, as well as theoretical and practical issues for consideration. 

A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions 

https://www.rwif.org/en/librarv/research/2009/12/a-practical-guide-for-engaging-stakeholders-in-developing- 

evalua.html 

This guide by the Robert Wood Johnson Foundation provides the reader with “a five-step process for involving 
stakeholders in developing evaluation questions and includes a set of four worksheets to facilitate this process.” This 
guide aims to assist evaluators and their clients in the process of engaging stakeholders—that is, those with a stake 
or interest in the program, policy, or initiative being evaluated. 

Developing an Effective Evaluation Report 

https://www.cdc.gov/eval/materials/Developing-An-Effective-Evaluation-Report TAG508.pdf 

This comprehensive workbook applies the CDC Framework for Program Evaluation in Public Health to report 
evaluation results to a variety of audiences. 
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Data Collection Methods: Pros and Cons 


Method 

Description 

Pros 

Cons 

Archival 

Data that have already been 
collected by an agency or 
organization and are in their 
records or archives 

• Low cost 

• Relatively rapid 

• Unobtrusive 

• Can be highly 
accurate 

• Often good to 
moderate validity 

• Usually allows for 
historical 

comparisons or trend 
analysis 

• Often allows for 
comparisons with 
larger populations 

• May be difficult to 
access local data 

• Often out of date 

• When rules for 
recordkeeping are 
changed, makes 
trend analysis 
difficult or invalid 

• Need to learn how 

records were 
compiled to assess 
validity 

• May not be data 
on knowledge, 
attitudes, and 
opinions 

• May not provide a 
complete picture 
of the situation 

Key 

Informant 

Interviews 

Structured or unstructured 

one-on-one directed 
conversations with key 
individuals or leaders in a 
community 

• Low cost (assuming 
relatively few) 

• Respondents define 
what is important 

• Rabid data collection 

• Possible to explore 
issues in depth 

• Opportunity to clarify 
responses through 
probes 

• Sources of leads to 

other data sources 
and other key 
informants 

• Can be time 
consuming to set 
up interviews with 
busy informants 

• Requires skilled 
and/or trained 
interviewers 

• Accuracy 
(generalizability) 
limited and 
difficult to specify 

• Produces limited 
quantitative data 

• May be difficult to 
analyze and 
summarize 
findings 
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Method 

Description 

Pros 

Cons 

Focus Groups 

Structured interviews with 
small groups of like individuals 
using standardized questions, 
follow-up questions, and 
exploration of other topics that 
arise to better understand 
participants 

• Low cost 

• Rapid data collection 

• Participants define 
what is important 

• Some opportunity to 
explore issues in 
depth 

• Opportunity to clarify 
responses through 
probes 

• Can be time 
consuming to 
assemble groups 

• Produces limited 
quantitative data 

• Requires trained 
facilitators 

• Less control over 
process than key 
informant 

interviews 

• Difficult to collect 

sensitive 

information 

• Accuracy 
(generalizability) 
limited and 
difficult to specify 

• May be difficult to 
analyze and 
summarize 
findings 

Surveys 

Standardized paper-and-pencil 
or phone questionnaires that 
ask predetermined questions 

• Can be highly 
accurate 

• Can be highly reliable 
and valid 

• Allows for 
comparisons with 
other/larger 
populations when 
items come from 
existing instruments 

• Easily generates 
quantitative data 

• Relatively high 
cost 

• Relatively slow 
design, 

implement, and 
analyze 

• Accuracy depends 
on who and how 
many people 
sampled 

• Accuracy limited 
to willing and 
reachable 
respondents 

• May have low 
response rates 

• Little opportunity 
to explore issues 
in depth 
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Potential Challenges to Obtaining Useful Data 


Some data sets may have collection or quality issues that will affect your ability to obtain useful 
data. In some cases, you can overcome these barriers by working with the data providers or 
your evaluator to reconfigure the data in ways that meet your needs. In other cases, you simply 
may not be able to use the data or will need to keep their limitations in mind when drawing 
conclusions based on the data. Caveats about data limitations, and its possible consequences 
for your analysis, should be included in data reports. 

Common barriers to obtaining useful data include the following: 

• Data may be aggregated. Hospitals, for example, often combine adult and youth data or 
data across several communities. This can be frustrating if you are seeking information 
about youth in your town. The agency may be able to sort the data for you. 

• Jurisdictions may overlap. For example, the jurisdiction boundary of your local police 
department may not correspond to that of the school district. A trauma center may draw 
patients from across your state. 

• Time periods may be inconsistent or too short. Data from one agency may be 
organized by calendar year, another by fiscal year, and another by school year. The data 
may not be current enough or collected for a long enough time to track trends accurately. 

• Data may be missing or incomplete. Information included in agency records and local 
data sets is often missing or incomplete. If the amount of missing data is large, the data 
may not provide an accurate picture of your community. This is especially true if some 
information is consistently missing, such as records from a particular school district or 
police precinct. Or, a failure to consistently record data (such as age or blood alcohol 
content) may make it impossible for you to analyze the data in ways that are useful for your 
efforts. 

• Data categories may not meet your needs. For example, sub-categories such as race or 
ethnicity are not always determined or implemented consistently across organizations. 
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Strategies for Conducting Effective Focus Groups 


The following guidelines related choosing participants for focus groups can help to generate 
information that is more reliable. 

Include People Who Can Provide the Information You Need 

Data collection involves asking the appropriate people for the appropriate information. Suppose 
you want to learn about parents’ attitudes and practices concerning teen alcohol use, and 
drinking and driving? You obviously want to ask parents. But you also might want to consider 
the following: 

• Should parents have children of a certain age in order to participate? (for example, no 
younger than 15?) 

• Do you want to include both mothers and fathers? 

• Does the ethnicity of the parents make a difference? 

• Should you include parents who drink and nondrinking parents? 


Try to define your participants as precisely as possible. It usually makes sense to consider 
gender, age, occupation, geographic location, ethnicity, and language. 

Include Participants Who Are Similar to One Another 

The less diverse your focus group, the better. If you want to gather information on Hispanic 
teenagers, teens who have recently emigrated from Somalia, and teens in the “heavy metal” 
subculture, organize individual focus groups for each category. 

There are two reasons for this: 

• An individual cannot represent a population. A focus group of 10 teenagers might not be 
able to provide a representative sample of all teens in your community. But it will probably 
generate more representative information than will one teenager included in a group 
spanning several generations. 

• Research shows that people are more likely to reveal their opinions and beliefs and to talk 
about sensitive issues when they are with people who they perceive to be like themselves. 
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Include Participants Who Do Not Know One Another 


Participants are more likely to be honest and forthcoming when they do not know the other 
people in the group. The following may occur when participants know one another: 

• They are less likely to reveal personal or sensitive information. 

• They are more likely to express views that conform to those of others in the group 
(especially others whom they perceive as having some power or influence outside the 
group). 

• They may respond to questions based on their past experiences with one another (which 
effectively reduces your sample size). 


Published: 08/06/15 
Last Updated: 09/04/2018 


Developed under the Substance Abuse and Mental Health Services Administration's Center for the Application of Prevention Technologies task 
order. Reference #HHSS283201200024I/HHSS28342002T. 



PREVENTION 

SOLUTIONS@EDC 


EDC 


Education 

Development 

Center 


Tips for Conducting Key Informant Interviews 

Although key informant interviews are more informal than other forms of data collection, they 
still require a structure to be effective. Your respondent is more likely to take you seriously 
(and provide better information) if you are prepared and the conversation has direction. 

Tips for conducting key informant interviews include the following: 

• Begin by introducing your project and purpose. Remind the respondent about your 
purpose and the ultimate use of the information. Also, explain who will have access to your 
interview notes and whether the respondents will be identified in any reports or public 
discussions of your investigation. 

• Start with an easy question. For example, ask how long your respondents have been in 
their jobs. This will set them at ease and provide a context for analysis (as someone who 
has been on the job for six months will not have the same perspective as someone who 
has been on the job for 10 years). 

• Ask your most important questions first. You might run out of time. This is especially 
important when interviewing people whose job might require them to end the interview early 
(such as emergency medical service or law enforcement personnel). 

• Ask the same (or parallel) questions of several respondents. For example, you might 
want to ask all respondents connected with a particular prevention program (or system) to 
list the three things they would like to see improved. Answers from a number of different 
people in a system can reveal programming obstacles or places in which the system needs 
to be improved. 

• Don’t move to a new topic prematurely. Don’t leave important issues hanging—you 
might run out of time before you can return to them. Also, you will get more useful 
information by discussing one subject at a time. 

• Be prepared to ask the same question in another way. Prepare several questions that 
try to elicit the same information. Turn to the alternate questions when your first question 
just doesn’t do the job. 

• Don’t get stuck on a question. Sometimes you just won’t get the information you want 
from a particular respondent. Know when to move on so you don’t frustrate yourself or 
antagonize your respondent by trying to elicit information that he or she does not have, 
cannot articulate, or isn’t willing to share. 

• Don’t let the interview go much over an hour. The people you chose as key informants 
are likely to be busy. The quality of the conversation can deteriorate if they feel rushed. 
Many of your respondents may be people with whom you might want to collaborate with in 
the future, so don’t antagonize them by letting an interview go on too long. 

• Record the interview if possible. And take notes. As with focus groups, transcribe the 
recording and type up your notes as soon as possible after the interview is completed. 

Don’t forget to get the respondent’s permission to make an audio recording. 
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Protective Factors: Adolescence through 
Young Adulthood 

Prevention is not just about eliminating a negative behavior; it is also about striving to 
optimize well-being and supporting factors that protect against misuse. These protective 
factors can reduce the negative impact of risk factors. The following tables, compiled by 
the National Research Council and Institute of Medicine and the United States Surgeon 
General, show select protective factors that are associated with healthy development at 
the individual, family, and school/community levels during specific stages of 
development from adolescence through young adulthood. 


PROTECTIVE FACTORS: ADOLESCENCE 


Individual 

Family 

Community (School) 

• Positive physical 
development (good health 
habits, good health risk 
management skills) 

• Positive intellectual 
development (life, school, 
vocational skills; critical 
and rational thinking; 
cultural knowledge and 
competence) 

• Positive psychological and 
emotional development 
(self-esteem and self¬ 
regulation; coping, 
responsibility, problem¬ 
solving; motivation and 
achievement; morality and 
values) 

• Positive social 
development 
(connectedness to peers, 
family, community; 
attachment to institutions) 

• Physical and psychological 
safety 

• Appropriate structure 
(limits, rules, monitoring, 
predictability) 

• Supportive relationships 
with family members 

• Opportunities to belong 
(sociocultural identity 
formation, inclusion) 

• Positive social norms 
(expectations, values) 

• Support for efficacy and 
mattering, or the feeling 
that one is making a 
difference 

• Opportunities for skill 
building 

• Integration of family, 
school, and community 
efforts 

• Physical and psychological 
safety 

• Appropriate structure 
(limits, rules, monitoring, 
predictability) 

• Supportive relationships 

• Opportunities to belong 
(sociocultural identity 
formation, inclusion) 

• Positive social norms 
(expectations, values) 

• Support for efficacy and 
mattering 

• Opportunities for skill 
building 

• Integration of family, 
school, and community 
efforts 
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PROTECTIVE FACTORS: YOUNG ADULTHOOD 


Individual 

Family 

Community (School/Work) 

• Identity exploration in love, 
work, and worldview 

• Subjective sense of adult 
status in self-sufficiency, 
making independent 
decisions, and becoming 
financially independent 

• Future orientation 

• Achievement motivation 

• Belief in a higher being, or 
involvement in spiritual 
practices or religious 
activities 

• An individual’s belief that 
they can modify, control, or 
abstain from substance 

use 

• Balance of autonomy and 
relatedness to family 

• Behavioral and emotional 
autonomy 

• Married or living with a 
partner in a committed 
relationship who does not 
misuse alcohol or drugs 

• Opportunities for 
exploration in work and 
school 

• Connectedness to adults 
outside of family 
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Chapter 4: How to Build Capacity to Prevent 
Drug Misuse on Your Campus 

Worksheet: Identifying New Partners 
Worksheet: Analyzing Existing Partnerships 
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PREVENTION COLLABORATION IN ACTION 

Identifying Needs and 
Opportunities for Collaboration 


Worksheet: Identifying New Partners 

Local stakeholders are key to the success of prevention efforts: they bring specialized knowledge, 
access to data, insight about priority populations, and a variety of other resources. Yet figuring out 
which organizations, agencies, or individuals to cultivate as partners can be challenging. There are 
many remarkable individuals and groups in your community, but not all of them will be a good fit for 
your prevention efforts at this time. 

This worksheet will help you record the gaps in your current prevention resources, develop a list of 
community stakeholders who might fill these gaps, and identify other potential stakeholders with whom 
you may want to partner in the future as new priorities and/or needs emerge. Once you complete this 
worksheet, you can use your list to prioritize which partners to engage. 

Step 1 : Answer these questions: 

1. Which prevention resources 1 do you currently need to strengthen or sustain your prevention 
efforts? (If you completed the worksheet Analyzing Existing Partnerships, you may want to 
review the resources you already have in place. To identify current resource gaps, review the tool 
What Do We Mean by Resources? ) 

2. Which new stakeholders from your community might help you fill identified resource gaps? (Fora 
list of potential partners, review the tool 21st Century Partners in Prevention . ) 

Step 2: Use your responses to the questions above to complete the chart below. 

■ First, list your potential partners. 

■ Next, record the prevention resource gaps each partner might fill. 

■ Describe additional prevention resources each partner might provide. 

■ Outline the benefits each partner may experience by joining the collaboration. 

■ In the remaining rows, list any additional partners you may want to engage, accompanied by the 
prevention resources they offer. (See the list of stakeholder groups in Part II of the worksheet 
Analyzing Existing Partnerships to see if there are any key groups you are missing.) 


1 For this tool, prevention resources are defined as fiscal, human, organizational, or other assets that help you address 
identified prevention needs in your community. 
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Step 3: Once you’ve completed the chart, decide which partners to contact first. This will help to ensure your time and effort reaching out to 
them is spent wisely. For help prioritizing, see the worksheet Assessing the Readiness of Potential Partners to Collaborate . 


Potential Prevention 
Partner 

Resource Gap Filled 

What specific resource need or gap 
could this partner fill? 

Other Value Added for You 

Which additional prevention resources 
could this partner bring to the table? 

Value Added for Partner 

What are the benefits to the potential 
partner of collaborating with you? 

1 . 




2. 




3. 




4. 




5. 











Potential Prevention 
Partner 

Resource Gap Filled 

What specific resource need or gap 
could this partner fill? 

Other Value Added for You 

Which additional prevention resources 
could this partner bring to the table? 

Value Added for Partner 

What are the benefits to the potential 
partner of collaborating with you? 

6. 




7. 




8. 




9. 




10. 
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Identifying Needs and Opportunities for Collaboration 


Worksheet: Analyzing Existing Partnerships 

Before engaging in new collaborative efforts, it’s important to assess the status of current partnerships. Who currently sits at your prevention 
table? What skills and expertise do they bring? How is the relationship working out? 

This worksheet is designed to help you develop a brief inventory of with whom you are working and how. Analyzing existing partnerships will 
help you see and appreciate the value of current partners. It will also help you determine whether the partners you have on board are the 
right ones, given your current prevention priorities, and to see where you may need to recruit new partners to fill identified gaps and/or 
ensure representation from key stakeholder groups. (For more on this, see the worksheet Identifying. New Partner .) 


Instructions 

■ List your current partners. For this tool, partners are defined as organizations or individuals with whom you share prevention resources. 

■ For each partner: 

» Write down the community sector this partner represents (i.e., Are there any sectors missing, given your prevention priorities?) 

» Describe the nature of your collaboration (i.e., How do you work together?) 

» Identify the prevention resources the partner contributes (i.e., How does collaborating with this partner strengthen your prevention 
efforts?) 

» Determine the partner’s current level of involvement (i.e., What is the nature of your collaboration?) 

» Describe the current status of your collaborative relationship (i.e., Is your collaborative relationship going well? Is this partner still 
relevant, given your current priorities? Would you like to move the relationship to the next level?) 
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Current Partner 

Sector 

Represented 

(e.g., parents, health 
care, business) 

Nature of the 
Collaboration 

(How do you and your 
partner work 
together?) 

Prevention 
Resources Shared 1 

(e.g., knowledge, skills 
access to priority 
populations) 

Level of 
Involvement 2 

(e.g., networking, 
cooperating, 
coordinating, full 
collaboration) 

Current Status 

(Is the collaboration 
going well/still 
relevant?) 

1 . 






2. 






3. 






4. 






5. 






6. 






7. 







1 

For this tool, prevention resources are defined as the fiscal, human, organizational, or other assets that help you address identified prevention needs 
in your community. See What Do We Mean by Prevention Resources? for a list of common prevention resources. 

2 See the tool Levels of Collaboration for descriptions of the different levels. 
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Current Partner 

Sector 

Represented 

(e.g., parents, health 
care, business) 

Nature of the 
Collaboration 

(How do you and 
your partner work 
together?) 

Prevention 
Resources Shared 

(e.g., knowledge, skills 
access to priority 
populations) 

Level of Involvement 

(e.g., networking, 
cooperating, 
coordinating, full 
collaboration) 

Current Status 

(Is the collaboration 
going well/still 
relevant?) 

8. 






9. 






10. 






11. 






12. 






13. 






14. 






15. 
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PREVENTION COLLABORATION IN ACTION 

Understanding the Basics 


Levels of Collaboration 

There are many different ways for stakeholders to work together. Collaboration between partners can range from informal (e.g., two 
agencies sharing information) to much more organized (e.g., multiple organizations working closely to achieve a shared vision). 

The following chart describes four levels of collaboration: networking, cooperation, coordination, and full collaboration. We suggest using the 
chart to determine your current level of involvement with a partner, as well as options for deepening this relationship over time. You can also 
use the chart to explore different options for collaborating with new partners. Please note that no single type of collaboration is “better” than 
another. The best type is the one that is the best fit, given what you and your partners hope to achieve. 



Networking 

Cooperation 

Coordination 

Full Collaboration 

0- 

Partners share information 

Partners support one 

Partners are engaged in 

With a formal agreement in 

to 

and talk with one another for 

another’s prevention activities 

mutual projects and 

place, partners work toward 


their mutual benefit. 

but have no formal agreement 

initiatives, modifying their own 

developing enhanced 

CO 

-C 


in place. 

activities to benefit the whole. 

capacity to achieve a shared 

£ 




vision. 


■ Loosely defined roles 

■ Somewhat defined roles 

■ Defined roles 

■ Formalized roles 


■ Loose/flexible 

■ Informal and supportive 

■ Formalized links, but 

■ Formal links, which are 

w 

£ 

relationships 

relationships 

each group retains 

written in an agreement 

3 

+-» 

■ Informal communication 

■ More frequent 

autonomy 

■ Frequent communication 

(0 

a> 

■ Minimal decision-making 

communication 

■ Regular communication 

■ Equally shared ideas and 

LL 

> 

■ No risk 

■ Limited decision-making 

■ Shared decision-making 

decision-making 

a) 


■ Little to no risk 

around joint work 

■ High risk but also high 




■ Low to moderate risk 

trust 




■ Share some resources 

■ Pooled resources 
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Networking 

Cooperation 

Coordination 

Full Collaboration 


■ Partners share what they 

■ Partners publicize one 

■ Partners serve together 

■ Partners sign a 


are doing to address 

another’s programs and 

on event planning 

memorandum of 


common community 

services. 

committees and 

understanding with each 

0 s " 

<D 

issues at interagency 

■ Partners write letters of 

community boards. 

other. 

_l 

meetings. 

support for one another’s 

■ Partners implement 

■ Partners develop 


■ Partners discuss existing 

grant applications. 

programs and services 

common data collection 

o 

o 

programs, activities, or 

■ Partners co-sponsor 

together. 

systems. 

—1 

services with other 

trainings or professional 

■ Partners care about the 

■ Partners participate in 

(/) 

organizations. 

development activities. 

same issues. 

joint fundraising efforts. 

0) 

o 


■ Partners exchange 


■ Partners pool fiscal or 

Q 


resources, such as 


human resources. 

cs 

f 


technology expertise or 


■ Partners create common 

£ 


meeting space. 


workforce training 



■ Partners attend one 


systems. 



another’s meetings and 





events. 
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PREVENTION COLLABORATION INACTION 


Engaging the Right Partners 


You Gotta Hear This! Developing an 
Effective Elevator Pitch 


There comes a time in every budding relationship where you will need to make the os/r--that is, 
request a commitment from a potential partner to do something. This tool is designed to help 
you develop a compelling argument, or "elevator pitch," for why they should say yes. 

And while not every ask will be made within the confines of an elevator, the term "elevator 
pitch" reminds us that these requests should be intentional and succinct, as if the elevator 
doors could open up at any moment. 

What Makes a Good Pitch? 

A good pitch anticipates and addresses the main questions a listener may have about what you 
are asking and why. Ultimately, every ask involves a dialogue—listening is critical to understand 
the needs of potential partners and developing a relationship. But before that, you need to 
make your case. To do that, you need a pitch that describes: 

The three W's— the "who," "what" and "why" of your message or 
request. 

The benefits of collaboration—to both you and your partner. 

Any barriers you've identified that could make it challenging to work 
together, and how you plan to get past them. 

A call to action that clearly identifies what you are asking potential 
stakeholder to do if they say "yes." 

Remember, a pitch that is effective in engaging one potential partner will not necessarily be 
right for another. Quality pitches are audience-specific, tailored to address the audience's 
unique needs, values, and priorities. 
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Preparing Your Pitch 


Here are some questions to help you develop a pitch that includes the key elements presented 
above: 


Pitch Element 

Questions to Help You Get There 

(Ww) Three w ' s 

Who are you and who do you represent? 

What do you or your organization do? 

Why is your organization doing this work? What 
need are you serving? 

Benefits & 

(/ Barriers 

How will their participation benefit you and your 
organization? What is the value-added to the 
potential partner of collaborating with you? 

What are potential barriers to working together? 

How will you address these barriers? 

Call to Action 

What specifically are you asking their organization to 

do? 

What are the immediate next steps that you would 

like them to take? 


As you develop your pitch, also make sure to: 

• Consider any recent political, social, and/or economic events that might influence 
your partner's receptivity to your pitch. Pitches are not made in a vacuum, so it's 
important to look at community context and climate, as these can affect how your pitch 
might be interpreted. 

• Avoid jargon. "Insider" language—that is, terms, phrases, or acronyms specific to your 
organization or sector—can may be confusing (at best) and off-putting (at worst). Take 
time to consider alternative ways to get your point across. 
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Here's an Example 


Here's an elevator pitch made to a business community leader, by a prevention task force chair. 
How it might be different if the Chair were approaching the Chief of Police? 


Pitch 

Component 


What She Said 


Why She Said It This Way 



Three W's 


My name is Terry Jones and I'm the task force 
leader for the Everytown Substance Abuse 
Coalition. Our mission is to reduce growing 
rates of alcohol use among the young adults in 
our community. These rates are particularly 
alarming for young adults who are in the 
workforce, where problem alcohol use often 
leads to poor job performance, as well as 
onsite alcohol-related injuries. 


Terry clearly defines who 
she is, the purpose of the 
coalition, and the link 
between alcohol use and 
workplace injury. 



Benefits & 
Barriers 


To prevent substance use in this group, we 
need to go where the young people are— 
where they play and where they work. And 
that's where you can really help us. You are a 
major employer of young people, so you can 
really help us reach them. I understand that as 
a business owner, you have limited time to 
dedicate to programs like ours. However, I 
think that engaging in this work could prove 
good for business, by reducing staff turnover 
as well as alcohol-related injuries at your 
stores. 


Terry knows that this 
business leader's biggest 
concern is having his 
workers injured while on 
the job, so she makes sure 
to include this in her pitch. 


Call to Action 


Would you be willing to partner with us to 
develop a series of trainings that we could 
offer to the young workers in your stores? We 
would do the heavy-lifting—we would just ask 
you to let us spend some time interviewing 


Terry is specific about how 
she'd like to work with this 
business leader. 
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Pitch 

Component 

What She Said 

Why She Said It This Way 

Call to Action 

(Cont.) 

some of your employees, and then provide 
the space and time to deliver the trainings. 

If we work together to tackle this problem, 1 
believe our partnership can help your business 
thrive and assist you in becoming one of the 
community's top performing businesses. Your 

involvement would also tie in well with the 

work that the Mayor's Commission is doing to 
prevent drug use in the community. 

Terry also places the 
request in the context of 
other prevention efforts 
going on in the city, and 

underscores the 

importance of working 
together. 


Practice Makes Perfect 

Use the template below to create your own elevator pitch to a potential partner. Before 
delivering it, make sure to check the following: 

• Is your pitch sensitive to your potential partner's needs, values, and priorities? 

• Does your pitch contain jargon or words that are difficult to understand? 

• Does your pitch take into account recent political, social, or economic events? Revise 
your pitch accordingly, and then go put it into use! 


Key Component 

Leading Questions 

Your Pitch 

(WWW) 

Who are you and who do you 
represent? 

What do you or your 
organization do? 


Three W's 

Why do you do this work? What 
need are you serving? 
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Key Component 

Leading Questions 

Your Pitch 

Benefits & 

Barriers 

What is the value-added to the 

potential partner of collaborating 
with you? 

How will their participation 
benefit you and your 
organization? 

What are potential barriers to 
working together? How will you 

address these barriers? 


Call to Action 

What specifically are you asking 
this person or organization to 

do? 

What are the immediate next 

steps that you would like them to 

take? 
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PREVENTION COLLABORATION IN ACTION 

Exploring Ways to Work Together 


Beginning Your Collaboration: 

Tips for a Safe and Satisfying Journey 

Think of a workgroup as a collection of people taking a journey together. As it plans its trip, members 

need to figure out where they’re going, how they will get there, and what route to take. 

■ A workgroup’s destination is its vision, dictating where the group wants to go. 

■ A workgroup’s vehicle is its structure and procedures. Its structure is the body; its procedures the 
engine. Members travelling in a broken-down vehicle (e.g., full of holes in the floor) are likely to “fall 
out,” or leave the group. And if the engine begins to fail, the journey is likely to slow down or come 
to a screeching halt. 

■ Lastly, the path or route a group takes is its goals and activities —how members will work together 
to get where they need to go. 

Building on this analogy, here are some tips for ensuring a smooth and safe collaborative ride! 

■ Create a shared vision. One of the first orders of business for collaborative workgroups is to 
discuss and create a shared vision. If members have different ideas about the ultimate purpose of 
the workgroup, the team will not be motivated to work toward common goals. A shared vision 
unifies the workgroup and makes it easier to figure out what needs to happen to make the vision a 
reality. 

■ Develop a well-defined structure. Whether the group’s structure is akin to a Mini Cooper or a 
charter bus, it’s important that everyone has somewhere to sit (i.e., a role), and that you make room 
for new members. Elements of a well-defined structure, such as clearly defined roles and 
responsibilities, meetings that begin and end on time, and regular progress updates will reduce 
potential frustration, keep members involved, and increase member satisfaction. 

■ Establish clear goals and related action steps. If you leave for a journey without a plan (whether 
GPS app or paper map), you are likely to get lost, waste energy (i.e., gas), and drive many extra, 
unnecessary miles. Groups that don’t have clearly-defined action steps connected to concrete goals 
can easily get “lost” in action. Busy team members whose activities are not tied to goals may be 
“spinning their wheels” and not actually moving the team’s prevention agenda forward. 
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■ Promote open communication during meetings. Create guidelines for participation or “ground 
rules” that support open, honest, and respectful exchanges. These serve as the “seat belts” that 
keep members safe, promote trust, and prevent tension and conflict among members. 

■ Use clear and transparent decision-making processes. Just as oil prevents an engine from 
seizing, clear and transparent decision-making processes will help to ensure your team doesn’t get 
mired in indecision. How the team approaches decision making is one of the first decisions it should 
make together! 

■ Be responsive to member needs. Like maintaining your car, so, too, must you attend to the needs 
of your members. Make sure that members find value in their participation—failing to do so is like 
driving all day and not stopping for food. One way to “feed” members is by providing trainings and 
in-services to build needed capacities. Another is to ensure that members can contribute in 
meaningful ways. Lastly, check in regularly with members to gauge their satisfaction with the group 
and find out if you need to do any tune-ups. 

■ Build the leadership capacity of members. This not only helps to keep members challenged and 
engaged, but will also contribute to the group’s sustainability. Having workgroup members who are 
involved in leadership roles is like having a spare tire—it ensures that your workgroup is not 
dependent on the involvement of a single individual to move forward. 

■ Assess progress regularly. Revisiting project goals, and associated roles and responsibilities, will 
help the team stay on track and avoid long detours. Keep in mind, however, that the shortest path 
to getting somewhere isn’t always the smoothest, or most scenic. Remember that the journey 
should also be fun and fulfilling. So acknowledge member contributions and celebrate your 
progress along the way! 
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Chapter 5: How to Plan a Successful Drug Misuse Prevention 
Program on Your Campus 

Worksheet: Developing Your Logic Model 
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Developing Your Logic Model: Worksheet 

A logic model is a visual tool that presents the logic, or rationale, behind a program or process. Substance abuse 
prevention practitioners use logic models to connect and communicate all of the elements of their prevention plans, and 
to guide their evaluation efforts. This worksheet presents a set of questions that practitioners can use to inform the 
development of their programmatic logic models. To learn more about logic models, review the PowerPoint 
presentation Developing a Logic Model t° Guide Program Evaluation. . 


Step One: Describing the Need for Your Program 


What problems in the community will your program address? Be specific about the positive behaviors you want to 
see strengthened or the negative behaviors you want to see changed and target audience. 


Why did you choose these particular problems in your community over other problems? Why is it important? 
(Consider magnitude, trends, severity, and economic costs). 
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What conditions or factors in your community contribute to these problems? Consider factors specific to your 
community. (Note: This is your explanatory theory). 


Are there other efforts in the community that address these problems? If so, how well? If not, why? 


How does your program fit relative to other approaches in your community? Does your program or practice add 
anything different? 


Do you expect support from your community or organization for your program? Why or why not? 
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Summary Statement of your Community's Needs 


Step Two: Defining Goals 


What do you want to see changed in the long term? 


Write your goal statement(s) here. Remember to tie your goals directly to your need statement. 
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Step Three: Specifying Your Objectives 


What specific changes do you anticipate will result from participation in your program? (or exposure to your 
practice or policy)? 


Are you expecting changes at the individual, organizational, community or policy level? 


If individuals, then which individuals, and what about them (i.e., knowledge, attitudes, behavior), do you expect to 
see change? 


What will be the expected magnitude of these changes? (e.g., at least 80% of participating youth will report an 
increase in their appreciation of their tribal heritage) 
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What will be the expected magnitude of these changes? (e.g., at least 80% of participating youth will report an 
increase in their appreciation of their tribal heritage) 


When do you anticipate seeing these changes? 


Write your short-term objective(s) here. Remember to tie your objectives directly to your problem statement and 
to the contributing factors that you identified. 


Write your long-term objective(s) here. 
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Step Four: Outlining Your Program Activities 


List the activities that comprise your program. 


How will these activities address those factors that contribute to the problem (i.e., help you accomplish your goals 
and objectives)? These are your theories of change. 


Review your initial list of activities. 


Which of these activities are critical to program success (i.e., must be in place for 
your program to succeed)? 


Which of these are short-term activities? 
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Which of these are long-term activities? 


Step Five: Identifying People Who Care If Your Program Succeeds 


Who is responsible for implementing your program? 


Who else—other than staff—wants your efforts to succeed? 


Who in your community may want to see your program fail? 


Consider your program activities. Have you engaged the people you need in order for your program to succeed? If not, 
who is missing? How will you bring them on board? Consider creating a chart: 


Program Activity 

Who Will Make it Happen? 

Are They On- 
Board? 

If Not, How Can You Get 
Them There? 
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Program Activity 

Who Will Make it Happen? 

Are They On- 
Board? 

If Not, How Can You Get 
Them There? 






















Step Six: Choosing the Right Process Measures 


How will you know your activities happened, as planned? 


For each activity, list the kinds of information you need to determine if it was implemented as planned: 


Program Activity (from Step 5) 

Process Measure 














Developed under the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for the Application of 
Prevention Technologies contract. Reference #HHSS277200800004C. 


8 























Step Seven: Choosing Outcome Measures 


How will you measure program-produced changes? 


List your short-term outcome measures (these should correspond with your short-term objectives). 


List your long-term outcomes measures (these should correspond with your long-term objectives). 
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Logic Model Template 


Needs 

Goals 

Objectives 

Activities 



Short-term: 

Short-term: 



Long-term: 

Long-term: 


Stakeholders 

Process Evaluation 

Outcomes 

People Responsible for Your 
Intervention: 

Program Activity 

Process Measure 

Short-term: 





People Who Want to See Your 
Program Succeed: 







Long-term: 

People Who Want to See Your 
Program Fail: 
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What Are Core Components...and Why Do 
They Matter? 

Core components are the most essential and indispensable components of an intervention 
practice or program (“core intervention components”) or the most essential and indispensable 
components of an implementation practice or program (“core implementation components”). 

Core Components for Interventions 

Part of an implementer’s goal is to implement only those attributes of a program or practice that 
are replicable and add value. Core intervention components are, by definition, essential to 
achieving good outcomes for consumers. However, understanding and adhering to the 
principles of intervention underlying each core component may allow for flexibility in form (the 
intervention’s processes and strategies) without sacrificing the function associated with the 
component. 

Knowing the core intervention components may allow for more efficient and cost-effective 
implementation and lead to decisions about what can be adapted to suit local conditions. Core 
intervention components may best be defined after a number of attempted applications of a 
program or practice, not just the original one. 

Core Components for Implementation 

The goal of implementation is to have practitioners base their interactions with clients and 
stakeholders on research findings (evidence-based practices and programs). Core 
implementation components help accomplish this task. The core implementation components 
consist of the following: 

• Staff selection: Beyond academic qualifications or experience factors, certain practitioner 
characteristics are difficult to teach in training sessions so must be a part of the selection 
criteria. Staff selection also represents the intersection with a variety of larger system 
variables. General workforce development issues, the overall economy, organizational 
financing, the demands of the evidence-based programs in terms of time and skill, and so 
on impacts the availability of staff for human service programs. 

• Pre-service and in-service training: Trainings are efficient ways to provide knowledge of 
background information, theory, philosophy, and values. They also help to introduce the 
components and rationales of key practices and provide opportunities to practice new skills 
and receive feedback in a safe training environment. 

• Ongoing consultation and coaching: Most of the skills people need can be introduced in 
training but really are learned on the job with the help of a consultant or coach. 
Implementation of evidence-based practices and programs requires behavior change at the 
practitioner, supervisory, and administrative support levels. Training and coaching are the 
principle ways in which behavior change is brought about for selected staff in the beginning 
stages of implementation and throughout the process of evidence-based practices and 
programs. 
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• Staff and program evaluation: Staff evaluation is designed to assess the use and 
outcomes of the skills that are reflected in the selection criteria, are taught in training, and 
reinforced and expanded in consultation and coaching processes. Assessments of 
practitioner performance and measures of fidelity also provide useful feedback to managers 
and implementers regarding the progress of implementation efforts and the usefulness of 
training and coaching. Program evaluation assesses key aspects of the overall 
performance of the organization to help assure continuing implementation of the core 
intervention components over time. 

• Facilitative administrative support: This provides leadership and makes use of a range 
of data inputs to inform decision-making, support the overall processes, and keep staff 
organized and focused on the desired outcomes. 

• Systems interventions: These are strategies that work with external systems to ensure 
the availability of the financial, organizational, and human resources required to support the 
work of the practitioners. 

These are interactive components that can compensate for one another so that a weakness in 
one component can be overcome by strengths in other components. Organizations are dynamic 
and there is an ebb and flow to the relative contribution of each component to the overall 
outcomes. The feedback loops are important in keeping the evidence-based program “on-track.” 
If the feedback loops (staff or process evaluations) indicate needed changes, then the system 
needs to be adjusted to improve effectiveness or efficiency. 

Critical functions of implementation consist of practitioner training, coaching the practitioner on 
the job, regularly assessing fidelity, and using that information to improve the performance of 
practitioners who are selected for the position. 

Multilevel Influences on Successful Implementation 

The core implementation components are important in changing the behavior of practitioners 
and other personnel who are key providers of evidence-based practices within an organization. 
The core components are contained within and supported by an organization that establishes 
facilitative administrative structures and processes to select, train, coach, and evaluate the 
performance of practitioners and other key staff members; carries out program evaluation 
functions to provide guidance for decision-making; and intervenes in external systems to assure 
ongoing resources and support for the evidence-based practices within the organization. 

The core components must be present for the implementation to occur with fidelity and good 
outcomes. The organizational components must be present to enable and support those core 
components over the long term. And, all of this must be accomplished over the years in the 
context of variable but influential changes in governments, leadership, funding priorities, 
economic boom-bust cycles, shifting social priorities, and so on. 
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Organizational Change and Development 

Implementation of evidence-based practices and programs almost always requires 
organizational change. The elements often described as important to organizational change are: 

• Commitment of leadership to the implementation process 

• Involvement of stakeholders in planning and selection of programs to implement, to 
encourage buy-in and ownership during implementation and continuing operations and to 
keep negative forces at bay 

• Creation of an implementation task force made up of consumers, stakeholders, including 
unions and community leaders to oversee the implementation process 

• Suggestions for “unfreezing” current organization practices (including the use of external 
consultants or purveyors), changing those practices and integrating them to be functional, 
and then reinforcing the new levels of management and functioning within the organization 

• Resources for extra costs, effort, equipment, manuals, materials, recruiting, access to 
expertise, re-training for new organizational roles associated with implementation of an 
innovation 

• Alignment of organizational structures to integrate staff selection, training, performance 
evaluation, and on-going training 

• Alignment of organizational structures to achieve horizontal and vertical integration 

• Commitment of ongoing resources and support for providing time and scheduling for 
coaching, participatory planning, exercise of leadership, evolution of teamwork 


Reference 

Implementation Research: A Synthesis of the Literature - 2005 (PDF I 2 MB) at the 
National Institute on Drug Abuse (NIDA) Clinical Trials Network Dissemination Library 


Published: 08/06/15 
Last Updated: 09/04/2018 
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Evaluation Tool —Providing Evaluation Technical 
Assistance: Questions to Guide Evaluation 
Planning 

The following tables include a series of questions to help prevention practitioners plan 
for program evaluation. By responding to the proposed questions, practitioners can 
begin to think through how to select appropriate evaluation measures, address data 
collection challenges, and collect quality data. The questions are organized according to 
the following seven themes: 

• Defining the target population 

• Reviewing existing data 

• Selecting a sample 

• Increasing response rates 

• Addressing ethical considerations 

• Administering a quantitative survey 

• Following up and sharing results 

DEFINING THE TARGET POPULATION 


Prevention efforts should focus on a population that demonstrates risk for substance misuse based 
on data. During evaluation planning, it’s important to understand this targeted population and the 
changes you hope to expect as a result of programmatic efforts. 


Key Questions 

Follow-up Questions 

Considerations 

Which population 
group(s) are your 
prevention efforts 
targeting? 

• Is this population group at increased 
risk for substance misuse? 

• What characteristics place them at 
increased risk? 

• How did you determine that this 
population group was at increased 
risk? 

• Depending on data findings, target 
populations might include, for 
example, adolescents, emerging 
adults, college students, alcohol 
retailers, prescribers, or specific 
racial/ethnic groups. 


• Are there subgroups within your 
target population who are at even 
higher risk? 
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Key Questions 


Follow-up Questions 


Considerations 


How large is your 
population of 
interest? 

• Is it feasible to survey at the 
individual level or is your sample so 
large that it would make sense to 
survey at a higher level, such as the 
school level? 

• The larger the sample, the more 
expensive it becomes to collect the 
necessary data. 

• Power analysis can help determine 
the sample size required to detect an 
effect with confidence. It may be 
appropriate to consult with a 
statistician before selecting a sample. 
For more information on sample size, 
see Selecting a Sample. 

Are you expected to 
demonstrate change 
at the state or local 
level? 

• How many subrecipients are 
funded? 

• What is the total number of units 
from which you can potentially 
sample? 

• Some grantees may be working with 
every county in the state, or they may 
fund only one agency, county, or 
town. 

• Units can be, for example, individuals, 
schools, organizations, or 
communities. 

What changes do 
you expect to see in 
that population? 

• What performance outcome(s) are 
you measuring and monitoring? 

• Do you have access to measures 
that include indicators relevant to 
your outcomes? 

• For example: PFS grantees must 
submit at least six measures for each 
sub-recipient to the cross-site 
evaluation (PEP-C): three underage 
drinking and three prescription drug 

measures. 


REVIEWING EXISTING DATA 


Before deciding to collect original data (which can be resource intensive), it can be helpful to look 
into what data are already being collected and whether those sources can be used for evaluation 
purposes. 


Key Questions 

Follow-up Questions 

Considerations 

Are you able to 
monitor change in 
your population of 
interest using 
indicators from 
existing datasets? 

• Does the existing data include 
information at the level for which you 
want to show change (i.e., local, 
state, or national)? 

• What is the quality of the existing 
data? 

• Be sure to understand the 
psychometric properties of the survey 
and the methodology used to collect 
the data in order to determine the 
quality and rigor of the data set and 
have confidence in the findings. 
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Key Questions 


Follow-up Questions 


Considerations 


Are you able to 
monitor change in 
your population of 
interest using 
indicators from 
existing datasets? 

(cont.) 

• How accurately do these data 
represent the population you are 
trying to study? 

• Who is collecting the information? 

• Are data collected consistently in 
standard intervals (e.g., annually, 
biannually, every 3 years)? 

• Are the same questions asked each 
time the survey is administered? 

• Have the response categories 
remained consistent or have they 
changed? 

• Example of local data: school district 
suspension/expulsion data 

• Example of state-level data: state- 
level youth surveys and state-level 
adult surveys 

• Example of national-level data: U.S. 
census data to access population 
and demographic information about 
the country 

Will these datasets 
provide available 
baseline data on 
outcomes of interest 
in your timeframe? 

• What outcomes do you expect to 
change in your population of 
interest? 

• Over what time period are you 
expected to show change in those 
outcomes? 

• Does the timing of the existing data 
collection align with the timing of your 
reporting needs? 

• How long is the delay in accessing 
data? 

• Will you have access to all of the 
data you need with the demographic 
breakdowns required for reporting? 

• Do these data sources provide 
accurate indicators of outcomes? 

• Is the sample size large enough to 
produce a stable estimate? 

• Can the data be disaggregated in 
order to draw conclusions about the 
population of interest? 

• Examples of outcomes: prevalence 
of binge drinking, number of heroin 
deaths 
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Key Questions 


Follow-up Questions 


Considerations 


Are these datasets 
part of ongoing 
collection efforts that 
will provide (timely) 
future outcome data 
on your population of 
interest? 

• How often are data collected? 

• How sustainable is the data source? 

In other words, if you are considering 
using a student survey, you may 
want to ask stakeholders about the 
likelihood that this survey will 
continue to be used to collect data in 

the future and the likelihood that the 

dataset will be made available in the 
future. 

• Examples of ongoing data collection: 
annual town census, 11 th grade youth 
survey 

• Data may be collected weekly, 
monthly, biannually, or annually. The 
more data points, the better when 
showing trends over time. 

• If a survey is only administered once 
every two years, but yearly data are 
needed, then that dataset may not be 
appropriate. 

• Fiscal constraints can sometimes 
mean surveys are discontinued. It is 
important to check in with key 
stakeholders on the sustainability 
plans of existing surveys or 

measures. 

Do existing data 
sources provide 
information on 
sample 

representativeness 
and response rates? 

• Is the sample represented in the 
existing survey data set reflective of 
your community demographics? 

• Do you have a minimum response 
rate that you need to meet for grant 
requirements? 

• Response rates for national surveys 
are often available on websites or in 
methodological reports. For example, 
in 2015 the YRBSS 1 reported a 
school response rate of 69%, a 
student response rate of 86%, and 
an overall response rate 2 of 60%.' 

• If sample representativeness or 
response rate information is not 
provided, consider using other data 

sources. 

Will datasets provide 
(timely) baseline and 
subsequent data for 
a control or 
comparison 
population? 

• Some existing data sources may 
allow you to gather data for 
comparison purposes—in which you 
compare individuals or groups who 
have been exposed to intervention 
activities to others who have not 

• Consider comparing data (e.g., 

YRBSS, NSDUH, 3 BRFSS 4 ) from 
your state or community with data 
from another state, community, or the 
nation. 

• Comparisons should be made only 
with states or communities that share 


1 YRBSS: Youth Risk Behavior Surveillance System (https://www.cdc.gov/healthyyouth/data/yrbs/index.htm) 

2 Overall response rate = (number of participating schools/number of eligible sampled schools) x (number of usable 
questionnaires/number of eligible students sampled) 

3 NSDUH: National Survey on Drug Use and Health (https://www.samhsa.gov/data/population-data-nsduh) 

4 BRFSS: Behavioral Risk Factor Surveillance System (https://www.cdc.gov/brfss/index.html) 
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Key Questions 

Follow-up Questions 

Considerations 

Will datasets provide 
(timely) baseline and 
subsequent data for 
a control or 
comparison 
population? (cont.) 

(controlling for possible alternative 
explanations). 

key relevant characteristics (e.g., 
similar on demographics, core risk 
factors, and political and legislative 
climates). 


SELECTING A SAMPLE 5 


Once it is decided that original data will be collected, various decisions need to be made regarding 
choosing a sampling frame. 


Key Questions 

Follow-up Questions 

Considerations 

Will you collect data 
from all members of 
your target 
population (a 
census) or a sample 
of the target 
population? 

• How large is the entire population of 
interest? 

• Is it feasible and necessary to collect 
data from everyone considering that 
number? 

• For example, it may be feasible to 
collect census data from a whole 
grade within a school and include all 
the students, but it may not be 
feasible to collect data from all high 
school students within a district 

because of the size of the student 
body; therefore, surveying a random 
sample of high school classrooms 
may be more feasible if you want to 
capture data from multiple grades 
that is representative of the district 

Will a statistician 
help you create a 
sampling strategy? 

• Have you selected a population or 
subset that you’d like to target for 
data collection? 

• Examples of sampling strategies: 
stratified random sampling, cluster 
sampling, systematic sampling 


• Do you know a statistician who can 
help you think through your sampling 
needs? 



5 For further discussion on selecting a sample, see Sample Representativeness and Nonresponse Bias: Frequently Asked 
Questions. 
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Key Questions 


Follow-up Questions 


Considerations 


What is your 
sampling unit? 

• Are you interested in outcomes at the 
individual level? At the school level? 

At the community level? 

• For example, sampling units could be 
individual people, classrooms, church 
congregations, or young adults 
across a community. 

What steps will you 
take to recruit 
participants? 

• Where are you likely to find potential 
participants? 

• Where do your potential participants 
like to congregate? 

• What media platform might catch 
their eye and entice them to 
participate? 

• Include your target population in 
discussions (if possible) concerning 
recruitment efforts, because they’ll 
know the social norms and have 
insights to inform your methods. 

• If you are working with a specific 
population, it may be helpful to have 
someone from that community act as 
a liaison/facilitator. 

How large does your 
sample need to be? 

• Have you conducted a power 
analysis to determine the likelihood 
that your data collection activities will 
include enough participants to 
understand whether there is a 
significant effect? 

• Studies of outcomes with larger effect 
sizes, larger sample sizes, and 
higher significance levels (p<.01 vs. 
p<.05) are more likely to detect a 
statistically significant effect if the 
intervention works as intended. 

• There are many online calculators 
available to help conduct power 
analysis. Examples include: 
http://statpaqes.info/#Power, 
http://www.apower.hhu.de/. and 
http://www.ncss.com/software/pass/: 

however, these may not take into 
account all factors involved in your 
research. 

• It is recommended that you work 
closely with a statistician when 
conducting power analyses. 
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INCREASING RESPONSE RATES 6 


Often the greater the response rate, the more trust that the sample is representative. There are 
ways to plan ahead as well as methods that can be employed to increase survey response. 


Key Questions 

Follow-up Questions 

Considerations 

Do you plan to send 
out frequent 
reminders and/or 
additional surveys to 
respondents in order 
to increase 
participation? 

• What type of reminders would be 
most effective for respondents? Post¬ 
cards? Emails? Text messages? 

• What time intervals would be most 
effective for reaching your target 
population without overtaxing them? 

• Mail-back Surveys: Some general 
guidance regarding follow-up for 
mail-back surveys includes the 
following: After one week of 
nonresponse, send a reminder 
postcard; if still no response, send 
out the initial mail-out with a 
replacement survey 3 and 7 weeks 
after the initial mail-out.'' Additional 
postcards and/or follow-up postcards 
might be necessary. The actual 
schedule for follow-up depends on 
your resources and the level of 
response to each wave of the survey. 

• Online surveys: Consider sending up 
to three reminder emails for online 
surveys, personalizing each 
message, and including the average 
time to complete the survey."' 

Do you plan to 
provide incentives? 

• Will you compensate individual 
participants and/or organizations 
helping you access those individuals 
(e.g., schools)? 

• Some research participants may 
receive monetary compensation 
before or after completing a survey. 
Others may be given non-monetary 
incentives, such as a “free homework 
pass” for students or a free meal. 

• Consider connecting with an 
appropriate IRB to consider the 
ethical implications of the incentives 
to make sure participants do not feel 
coerced into participation. 


6 For further discussion on increasing response rates, see Sample Representativeness and Nonresponse Bias: Frequent /y 
Asked. Questions. 


Developed under the Substance Abuse and Mental Health Services Administration’s Center for the Application of 
Prevention Technologies task order. Reference #HHSS283201200024I/HHSS28342002T. March 2018 


PAGE 7 













Key Questions 


Follow-up Questions 


Considerations 


What information is 
useful for assessing 
the 

representativeness 
of the final sample? 

• What information will you be able to 
collect on non-respondents? 

• Does the final sample represent the 
target population proportionally? 

• For example, if you want to find out 
what percentage of males responded 
to the survey, you can compare the 
proportion of male non-responders to 
the proportion of males in the final 
sample. 

• Many times a sample should 
represent the larger population of 
interest demographically. For 
example, if 90% of the entire school 
district is female, but your school 
district survey responses include a 
sample of 20% females and 80% 
males, then there might be bias in 
your sampling strategy. 

What steps will you 
take to promote the 
survey? 

• What is your purpose for promoting 
the survey? 

• Will you involve the community to 
convey the value of the survey? 

• Who are key opinion leaders in the 
targeted community? 

• What media channels does the target 
population access most frequently? 

• Consider conducting focus groups or 
key informant interviews to determine 
what media channel(s) would be best 
to reach the targeted sample 
population (e.g., radio, T.V., social 
media). 

• Examples: Consider setting up a 
booth at a community event where 
you have information about the 
upcoming survey and intent for the 
research. It also may be helpful to 
write the local newspaper or campus 
newspaper to see if the survey could 
be explained in a news story. 
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ADDRESSING ETHICAL CONSIDERATIONS 


Ethical considerations must be taken into account when conducting evaluation, especially when 
collecting data from vulnerable populations such as youth. 


Key Questions 

Follow-up Questions 

Considerations 

What human 
protections 
requirements must 
be met before data 
collection can 
begin? 

• Will you be required to submit your 
data collection procedures and 
instruments to an institutional review 
board (IRB), a college/university 
research office, university board of 
trustees, college president, dean of 
students, department chair, or 
classroom instructors? Or, if working 
with Native American populations, 
the Tribal Council? 

• Design a protocol to ensure the 
confidentiality of data. 

• Design a protocol to make sure that 
all respondents understand the 
purpose of the survey and that they 
have a right to not participate. 


• If so, how long does the approval 
process take and what deadlines 
might be relevant for submission? 

Can the instrument be an 
amendment to an existing IRB- 
approved application? 


Are there any 
questions included 
in your data 
collection 

instruments to which 
community members 
might oppose? Are 
there questions 
about illegal 
behaviors? 

• Are there any questions on the 
survey that might be a “red flag” for 
key stakeholders or participants? 

• What steps will you take to engage 
key stakeholders and secure 
stakeholder buy-in for the survey? 

• What procedures can you put in 
place to ensure that participants feel 
comfortable providing sensitive 
information? 

• Examples: Some key stakeholders 
may not approve of asking youth 
about topics like sexual orientation, 
sexual behavior, suicide, or other 
sensitive topics. 

Will you be 
collecting data from 
one of the following 
population groups: 
youth underage 18 , 
prisoners, or military 
personnel (or any 
others that require 
special clearance)? 

• Who has the authority to grant 
permission to collect data in the 
target area (e.g., tribal council, school 
district, and/or school principal)? 

• Is there a special application process 
researchers are required to 
complete? If so, what is the 
timeframe for this process (e.g., 
weeks or months)? 

• Buy-in at the community level and 
from the population you are targeting 
(e.g., youth) can assist in successful 
data collection. 

• To conduct a school survey, 
permission may need to be granted 
through the school district 
superintendent, school board and/or 
principals. This may require making a 
presentation to the school board 
describing the purpose and intent of 
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Considerations 


Key Questions Follow-up Questions 


Will you be 
collecting data from 
one of the following 
population groups: 
youth under age 18, 
prisoners, or military 
personnel (or any 
others that require 
special clearance)? 

(cont.) 

• If you don’t obtain permission to 
collect certain types of data/survey 
certain populations, what alternative 
plans exist to meet any grant-specific 
reporting requirements? 

the data collection, as well as sharing 
drafts of survey items. It may also 
require notifying parents in advance 
of the survey to provide them with the 
opportunity to permit or refuse their 
child’s participation. 

Does your 
state/school/district 
require active 7 or 
passive 8 parental 
consent for 
surveying youth 
under age 18? 

• If you are collecting survey data from 
minors, how will you obtain parental 
consent? Keep in mind that passive 
consent might be allowed at the state 
or district level, but individual schools 
may require active consent. 

• Requirements regarding passive or 
active consent vary by state, by 
school district, and sometimes by 
school. Passive permission is usually 
preferred because it requires the 
fewest resources and does not often 
lead to many refusals. 

• Response rates may be substantially 
lower when active consent is 
required. This also requires many 
more resources to track which 
students have receiveved parental 
permission to participate. 

• It is important to create a plan for 
contacting parents and following up 
with them to address questions and 
make sure that they return the active 
consent form. 


7 Active consent: Requires distributing a form to parents explaining the study and having parents sign it in order to allow 
their child(ren) to participate. 

8 Passive consent: Requires distributing a form to parents explaining the study and having parents sign it if they refuse 
having their child(ren) participate. 
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ADMINISTERING A QUANTITATIVE SURVEY 


Administering a quantitative survey can be resource intensive, but there are ways to be efficient and 
ethical while still collecting high quality data. 


Key Questions 

Follow-up Questions 

Considerations 

Have you (or others 
you know of) 
collected data on 
outcomes of interest 
from the same 
population? 

• How often are these data collected? 

• Who collects it? 

• Do you have a good relationship with 
that individual, agency, department, 
or stakeholders? 

• For example, a survey that is 
administered through the state’s 
department of education may already 
include data on the outcomes of 
interest; therefore, it may be 
unnecessary to conduct another 
survey. 

• Data-sharing agreements may need 
to be formalized depending on the 
protocols to which the data owner 
needs to adhere. 

Can you build on 
other surveys that 
target population(s) 
may complete during 
your timeframe? 

• What surveys are currently being 
implemented with your target 
population? 

• What are the content of the surveys? 

• How long or burdensome are the 
surveys for participants? 

• How will time of data collection be 
managed so that staff and 
participants are not overburdened? 

• Consider adding items to an existing 
survey that the school district or other 
entity is conducting. This may not be 
an easy task and may require some 
negotiation and discussion with the 
survey originator. 

Is the population 
defined by, and 
therefore reached in, 
specific settings? 

• Are there specific places where the 
survey can be administered that are 
frequented by the target population? 

• Consider meeting the population at 
the locations they normally patron to 
administer a survey. For example, 
meet students at their school or 
college; meet community members at 
their faith-based organizations, out- 
of-school community groups, or 
clubs. 

• If you are working with specific 
populations, it may be helpful to have 
someone from that community act as 
a liaison/facilitator. 
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Key Questions 


Follow-up Questions 


Considerations 


Do you plan to use a 
standardized 
questionnaire 
already being used 
out in the field? 

• Does its use require permission? If 
so, do you have permission? Is there 
an associated fee? 

• Are the questions worded in the 
survey sensitive enough to capture 
change (i.e., lifetime vs. past-year vs. 
past 30-day substance use)? 

• For example, the Communities That 
Care Youth Survev and the YRBSS 
are free and readily available, while 
the Monitoring the Future Survey 
may not be used widely without 
permission from the developers. 

• If designing a new survey, try to 
create/craft items that parallel (or 
duplicate) the national surveys to 
allow some comparability of results, 
especially if the national survey is 
used as a baseline measure. It’s 
helpful to double check wording to 
make sure the same question is 
asked with the same response 
options, and that the national survey 
methodologies haven’t changed over 
time. 

What method will 
you use to 
administer your 
survey? 

• How will the mode of administration 
affect your ability to implement the 
survey and obtain an adequate 
response rate? 

• Methods may include: in-person 
interview, pencil-and-paper, audio- 
assisted computer, online, telephone 
interview, mail, or combination. In 
general, web-based surveys typically 
have lower response rates than in- 
person interviews iv and face-to-face 
interviews typically have higher 
responses than mail-in surveys. 

• Consider how choices about 
administration impact who is (and is 
not) included in your survey. For 
example, in-person administration in 
schools means high-risk students 
might be missing—like those with 
histories of truancy or those who 
have dropped out. Similarly, online 
surveys can impact who is (and is 
not) included, omitting those who lack 
access to technology. 
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Key Questions 


Follow-up Questions 


Considerations 


Is the survey written 
so that participants 
will understand the 
questions’ content? 

• What is the reading level or literacy 
skill of your target population? 

• Are the survey questions worded in a 
way that is easy for participants to 
understand? 

• Can accomodations be made to 
support members of the target 
population who may need extra 
assistance in completing the survey? 

• Reading level can be assessed using 
computer programs. Microsoft Word 
can generate a readability statistic 
that will report the Flesch-Kincaid 
Grade Level, which corresponds to 

U.S. school grade level. 

• For the general population, it is 
recommended that surveys not 
include items that require more than 

8 years of formal schooling and, for 
vulnerable populations, no more than 

5 years of formal schooling. v For 
example, the Massachusetts Youth 
Risk Behavior Survey, designed for 
high school students, was written at 
the 7 th grade reading level. vi 

Is the survey 
culturally competent 
for the intended 
participants? 

• Are you targeting immigrant 
population(s) or non-English 
speaking populations? 

• Will you need to administer the 
survey in languages other than 
English? 

• What translation services are 
available? 

• Translation of surveys requires 
multiple steps. A simple model 
includes 3 steps: (1) write survey in 
English; (2) translate survey into 
second language (translation); (3) 
translate survey from the second 
language back into English (back- 
translation). Once back translation is 
finished, the text is compared to the 
original survey. The translators meet 
to negotiate changes to ensure that 
the final translation matches the 
original English translation as closely 
as possible. vii 

• It may be helpful to consult a linguist. 

• Consider seeking input on 
questionnaire wording from a 
community advisory board or 
representatives of the target 
population to get feedback regarding 
cultural appropriateness. 

How much will it 
cost to collect data 
on outcomes of 
interest in your 
target population? 

• How much money and resources do 
you have available for data collection 
purposes? 

• The cost of data collection varies 
according to collection method. For 
example, face-to-face survey 
interviews are usually the most costly 
because they require training survey 
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Key Questions 


Follow-up Questions 


Considerations 


How much will it 
cost to collect data 
on outcomes of 
interest in your 
target population? 

(cont.) 

• How many participants do you need 
to survey and what response rate is 
expected? 

• What quality of data is sufficient? 

administrators and perhaps paying 
them for their time. Web-based 
survey methods may cost less (due 
to free to low-cost online programs) 
but the quality of the data and 
response rates may be lower. For 
more on the pros and cons of 
different data collection methods, 
visit: 

https://preventionsolutions.edc.orq/ 

services/resources/data-collection- 

methods-pros-and-cons 

Who will conduct 
data collection 
activities? 

• Has this individual (or individuals) 
been trained in data collection? 

• Is additional training needed? 

• Interviewers need to be trained on 
data collection processes, as well as 
on the human protection protocol. 

They may also need refresher 
trainings throughout the data 
collection process if there are 
changes to the survey or procedures. 

Who will take 
primary 

responsibility for 
developing and 
ensuring that the 
data collection 
protocols are 
followed? 

• Are the data collection protocols 
documented and standardized? 

• Are the protocols clear, 
understandable, and easy to follow? 

• Have data collectors been 
adequately trained in data collection 
protocols? 

• Have the instruments and protocols 
been approved by an Institutional 
Review Board (IRB), if necessary? 

• If possible, hire a data collection 
coordinator to make sure that data 
collection procedures are followed 
and standardized. 

• Periodically during data collection, 
review randomly selected surveys to 
determine completeness. Conduct 
routine check-in meetings with 
interviewers or data collectors to 
ensure they are following protocol, 
and brainstorm solutions to potential 
barriers to following protocol. 

What measures will 
you take to assure 
respondents that 
their responses will 
be kept private? 

• Are you asking questions about 
illegal behaviors? 

• Are you asking other sensitive 
questions, such as about suicidal 
thoughts and behaviors, or 
experiences of abuse (emotional, 
physical, and/or sexual)? 

• What protocols do you have in place 
to protect the confidentiality of 
participants and their responses? 

• Consider connecting with an 
appropriate IRB (may be required) to 
review the methods and measures in 
order to ensure the protection of the 
participants. 

• Instead of names, pair each 
participant with a unique identifier 
(code) marked on their survey form. 

• Keep the list of paired names and 
identifiers stored in a locked cabinet 
or password-protected file and 
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Key Questions 


Follow-up Questions 


Considerations 


What measures will 
you take to assure 
respondents that 
their responses will 
be kept private? 

(cont.) 


separate from the surveys—and limit 
who has access to this list. 

Train interviewers on research ethics 
and human protection protocols and 
ensure they are followed. 

Require password protection and 
security software for the computers 
used for tracking participation and 
data input. Only allow password 
access to necessary people who are 
trained in confidentiality and ethics. If 
possible, store data only on external 
hardware and lock up the hardware 
in a secured cabinet and room when 
not in use. 


FOLLOWING UP AND SHARING RESULTS 


In order to make evaluation respectful, useful, and purposeful, decisions need to be made regarding 
the dissemination of evaluation results. 


Key Questions 


Follow-up Questions 


Considerations 


Will results be 
shared with key 
stakeholders? 


• In what format will key stakeholders 
be most receptive to reading and 
understanding evaluation results? 

• How can you best convey 
program/strategy successes as well 
as barriers/challenges? 

• When would it be best to share 
results? 


• Evaluation results can be used to 
improve and strengthen programs or 
strategies. Consider discussing a 
process with key stakeholders for 
realigning resources or strategies to 
put the evaluation results to 
beneficial use. 

• Sharing evaluation results with 
stakeholders helps to build 
relationships and trust. This is 
particularly important when working 
with minority populations, such as 
Native American/Alaska Native 
populations or lesbian, gay, bisexual, 
or transgender populations. 
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Key Questions 


Follow-up Questions 


Considerations 


What type of follow¬ 
up are you prepared 
to implement with 
participants? 

• Will you provide survey results to 
participants? 

• How will you thank participants? 

• Consider sending a thank you via 
letter or email or postcard. 

• Consider sharing some of the key 
results from the study so that 
participants feel that there will be 
positive outcomes as a result of their 
involvement. 

How will evaluation 
results be used? 

• What process will you follow to 
realign resources to improve, 
replace, or augment strategies or 
programs based on the evaluation 
results? 

• Reflect on process evaluation 
measures that document 
implementation and see if 
adaptations were associated with 
more or less positive outcomes. 


' Kann, L., McManus, T., Harris, W. A., Shanklin, S. L., Flint, K. H., Hawkins, J., ... & Zaza, S. (2016). Youth 
Risk Behavior Surveillance — United States, 2015. MMWR Surveillance Summaries, 65(6), 1-174. 

" Hoddinott, S. N., & Bass, M. J. (1986). The Dillman Total Design Survey Method: A sure-fire way to get high 
survey return rates. Canadian Family Physician, 32, 2366-2368. Retrieved from: 
http://www.ncbi.nlm.nih.aov/pmc/articles/PMC2328022/pdf/canfamphvs00201-0076.odf 

Monroe, M. C., & Adams, D. C. (2012). Increasing response rates to web-based surveys. Journal of 
Extension, 50(6), 6-7. 

iv Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? 
Assessment & Evaluation in Higher Education, 33(3), 301-314. 

v Paz, S. H., Liu, H., Fongwa, M. N., Morales, L. S., & Hays, R. D. (2009). Readability estimates for commonly 
used health-related quality of life surveys. Quality of Life Research, 18(7), 889-900. Retrieved from: 
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2724639/ 

vi Massachusetts Department of Education. (2006). Introduction & survey methods. In: 2005 Massachusetts 
Youth Risk Behavior Survey (pp.1-5). Retrieved from: 
http://www.doe.mass.edu/cnp/hprograms/vrbs/05/ch1.pdf 

vii Bernard, H. R. (2011). Research methods in anthropology: Qualitative and quantitative approaches. 
Lanham, MD: AltaMira Press. 
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Reporting Your Evaluation Results 

Evaluation results are used to improve programs, sustain positive outcomes, and improve the 
community’s overall plan for addressing substance misuse and 

promoting wellness. But they can be used for other reasons as well, such as to help obtain 
funding and to build community awareness and support for prevention. 

So evaluation results need to get into the hands of the people who can use them. Keep in mind 
that organizations don’t use evaluation results; people do. The Department of Health, for 
example, isn’t going to use the results of an evaluation, but “Cathy Smith” in the Department of 
Health may. So, unless you get the results of the program evaluation into her hands and 
explain how she can use the results, they will sit on a shelf somewhere in the Department of 
Health. 

Follow these general guidelines for reporting your results: 

• Brief stakeholders throughout the process. Try to avoid surprising your stakeholders with the 
results of your evaluation. Brief them along the way, rather than waiting until the end of the 
project. Present a draft form of your report before it goes public. 

• Create a dissemination plan. Identify the various audiences that need to see the results 
(including the population the intervention focuses on), what information would be most 
useful to them, and how to get it into their hands. 

• Select formats for reporting results. Be sure to use the most appropriate format for each 
audience (for example: public presentation, social media, flyers, reports). 

• Help stakeholders understand the data. Take time to review the findings with your 
stakeholders, discussing the ramifications of what you found. Don’t shy away from negative 
or unexpected results. Instead, use these as an opportunity to inform future efforts. 

Remember that each stakeholder has his or her own interests and may need different kinds of 
information about the results of an evaluation. So, one size will not fit all when sharing 
evaluation results with stakeholders. 

The following set of questions can guide how evaluation results are presented, in order to 
ensure that results are relevant to various stakeholders and community members: 

1. WHAT data have you collected? 

2. WHY do you want to share the data? 

3. WITH WHOM will you share the data? 

4. WHAT data would be most useful? 

5. HOW are you going to present the data? 

6. WHO will present the data? 

7. WHERE will the data be presented? 

8. WHEN will the data be presented? 
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Answering these questions will help you determine the presentation of your evaluation results 
to various stakeholders. 


Some factors may influence whether and how your evaluation results get used. So keep these 
in mind: 

• The way in which findings are reported, including layout, readability, and user-friendliness, 
all make a difference. The timing is also critical. If a report is needed for a legislative 
session, but isn’t ready in time, then the chances of the data being used drop dramatically. 

• The quality of the evaluation and relevance of the findings matters. If the evaluation design 
is logically linked to the purpose and outcomes of the project, the findings are far more likely 
to be put to use. 

• The availability of support and technical assistance, after findings are reported, can sustain 
use. Questions of interpretation will arise over time, and people will be more likely to use the 
results if those kinds of questions can get answered. 

• The political context or climate can have an impact. Some evaluation results will get used 
because of political support, and others will get squashed because of political pressure. 

• Other factors, like the size of your organization or program, may matter as well. Sometimes 
larger programs get more press. Sometimes targeted programs do. 

• Consider competing information. For example, are there results from similar programs that 
confirm or deny your results? Are there other topics competing for attention? 
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Selecting an Appropriate Evaluation Design 


The first step in any project is to develop a plan for getting the work done. The plan for an 
evaluation project is called the “design.” 

All too often, prevention practitioners launch into their evaluation without coming up with a plan. 
They start thinking about how to collect data before determining what to collect. This is usually 
accompanied by the phrase, “Let’s do a survey!” But before choosing methods, practitioners 
need to back up. 

Designing an evaluation is a process that starts out general, but which ultimately becomes very 
specific. The first step is to clarify the purpose of the evaluation. That leads to developing 
questions that then require information and data obtained from methods. But the methods come 
last, not first. 

One of the primary purposes of the evaluation is to determine if the program or intervention had 
the desired effect. A classic research study found that individual behaviors and workers’ 
performance can improve simply because they know they are part of the study. The “Hawthorne 
Effect,” as this phenomenon is known, is a thorn in the side of evaluators. 

To correct for the Hawthorne Effect, some type of comparison generally needs to be made to 
make sure the change was the result of the intervention, and not due to the attention received. 
There is a continuum of evaluation rigor among these different methods of comparison. 

Experimental Design 

Typically, the most rigorous evaluation approach is an experimental design where participants 
are randomly assigned to a program or a control group. Those participants in the program 
group receive an intervention, while those in the control get either the existing program or in 
some cases, no program. Some type of pre/post assessment is provided to both groups and the 
results are then compared to determine if there were differences between groups. This method 
provides the greatest support for ruling out plausible explanations. 

Quasi-experimental Design 

Similar in structure to the experimental design, the quasi-experimental design does not rely on 
random assignment in making assignments to a program or comparison group. A quasi- 
experimental design is frequently used when there is not a sufficient number of participants 
available to randomly assign to a program and control group. As a result, in a quasi- 
experimental design, a significant challenge is identifying comparison groups and then 
collecting data from them. To score well on most federal lists of evidence-based programs, it is 
important, at a minimum, to use a quasi-experimental design. 
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Factors to Consider In Choosing an Evaluation Design 


The Purpose of the Evaluation 

The purpose of the evaluation varies greatly from program to program. As discussed previously, 
evaluation can be defined as the systematic collection of information about program activities, 
characteristics, and outcomes to reduce uncertainty, improve effectiveness, and make 
decisions. Most frequently, the emphasis is on whether or not the program had the intended 
effect. But, information could be collected that focuses only on the number of individuals served. 
Or evaluation data may be collected to use for marketing purposes. Each of these different 
strategies would require differing evaluation designs and differing evaluation skills. 

What Will Be Evaluated 

Is it the entire project or only certain components? For example, if you are part of a coalition, 
you may think about evaluating individual coalition initiatives like the provision of after-school 
activities. But don’t forget that you may also want to evaluate changes in the coalition over 
time—things like growth of coalition membership, formal agreements with key community 
organizations—and the impact of the coalition on the overall community. 


Who Wants to Know What 

Keep all of your stakeholders in mind. Program providers may want to know what’s working and 
what isn’t. Funders may want to know if the program is cost-effective and supported in the 
community. They may also have specific measures that are required. Communication between 
funders, project staff, and evaluators is essential to ensure the necessary data are identified, 
collected, analyzed, and reported in a manner that is understood from the beginning. 


When Results Are Needed 

An evaluation is often bound by schedules and deadlines that are beyond your control. Think 
about school calendars and funding cycles as examples. If your reporting needs are short-term, 
don’t ask questions that require long-term follow-up. Process information is generally needed 
quickly. Short-term outcome results often need to be reported back in a timely manner (usually 
within 6 to 12 months of program implementation), while more long-term results are typically not 
available until sometime after program completion (often 3 to 5 years). 


What Will Be Done with the Evaluation Results 

Think broadly about the utility of your results. You can use your data not only to meet funding 
requirements, but also to garner community and school support, inform future planning or 
programming, support community stakeholders, etc. But, the degree of influence a program has 
on the evaluation findings is largely dependent on the research design used. 
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Strategies for Conducting Effective Focus Groups 


The following guidelines related choosing participants for focus groups can help to generate 
information that is more reliable. 

Include People Who Can Provide the Information You Need 

Data collection involves asking the appropriate people for the appropriate information. Suppose 
you want to learn about parents’ attitudes and practices concerning teen alcohol use, and 
drinking and driving? You obviously want to ask parents. But you also might want to consider 
the following: 

• Should parents have children of a certain age in order to participate? (for example, no 
younger than 15?) 

• Do you want to include both mothers and fathers? 

• Does the ethnicity of the parents make a difference? 

• Should you include parents who drink and nondrinking parents? 


Try to define your participants as precisely as possible. It usually makes sense to consider 
gender, age, occupation, geographic location, ethnicity, and language. 

Include Participants Who Are Similar to One Another 

The less diverse your focus group, the better. If you want to gather information on Hispanic 
teenagers, teens who have recently emigrated from Somalia, and teens in the “heavy metal” 
subculture, organize individual focus groups for each category. 

There are two reasons for this: 

• An individual cannot represent a population. A focus group of 10 teenagers might not be 
able to provide a representative sample of all teens in your community. But it will probably 
generate more representative information than will one teenager included in a group 
spanning several generations. 

• Research shows that people are more likely to reveal their opinions and beliefs and to talk 
about sensitive issues when they are with people who they perceive to be like themselves. 
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Include Participants Who Do Not Know One Another 


Participants are more likely to be honest and forthcoming when they do not know the other 
people in the group. The following may occur when participants know one another: 

• They are less likely to reveal personal or sensitive information. 

• They are more likely to express views that conform to those of others in the group 
(especially others whom they perceive as having some power or influence outside the 
group). 

• They may respond to questions based on their past experiences with one another (which 
effectively reduces your sample size). 
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Tips for Conducting Key Informant Interviews 

Although key informant interviews are more informal than other forms of data collection, they 
still require a structure to be effective. Your respondent is more likely to take you seriously 
(and provide better information) if you are prepared and the conversation has direction. 

Tips for conducting key informant interviews include the following: 

• Begin by introducing your project and purpose. Remind the respondent about your 
purpose and the ultimate use of the information. Also, explain who will have access to your 
interview notes and whether the respondents will be identified in any reports or public 
discussions of your investigation. 

• Start with an easy question. For example, ask how long your respondents have been in 
their jobs. This will set them at ease and provide a context for analysis (as someone who 
has been on the job for six months will not have the same perspective as someone who 
has been on the job for 10 years). 

• Ask your most important questions first. You might run out of time. This is especially 
important when interviewing people whose job might require them to end the interview early 
(such as emergency medical service or law enforcement personnel). 

• Ask the same (or parallel) questions of several respondents. For example, you might 
want to ask all respondents connected with a particular prevention program (or system) to 
list the three things they would like to see improved. Answers from a number of different 
people in a system can reveal programming obstacles or places in which the system needs 
to be improved. 

• Don’t move to a new topic prematurely. Don’t leave important issues hanging—you 
might run out of time before you can return to them. Also, you will get more useful 
information by discussing one subject at a time. 

• Be prepared to ask the same question in another way. Prepare several questions that 
try to elicit the same information. Turn to the alternate questions when your first question 
just doesn’t do the job. 

• Don’t get stuck on a question. Sometimes you just won’t get the information you want 
from a particular respondent. Know when to move on so you don’t frustrate yourself or 
antagonize your respondent by trying to elicit information that he or she does not have, 
cannot articulate, or isn’t willing to share. 

• Don’t let the interview go much over an hour. The people you chose as key informants 
are likely to be busy. The quality of the conversation can deteriorate if they feel rushed. 
Many of your respondents may be people with whom you might want to collaborate with in 
the future, so don’t antagonize them by letting an interview go on too long. 

• Record the interview if possible. And take notes. As with focus groups, transcribe the 
recording and type up your notes as soon as possible after the interview is completed. 

Don’t forget to get the respondent’s permission to make an audio recording. 
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Using Process Evaluation to Monitor Program 
Implementation 

Process evaluation involves analyzing how program activities are delivered. Prevention 
practitioners seek to find the answers to these central questions: 

• Who delivers the program and how often? 

• To what extent was the program implemented as planned? 

• How is the program received by the target group and program staff? 

• What are barriers to program delivery? 

• Was the data used to make program improvements/refinements? If so, what changes were 
made? 

These questions enable practitioners to also assess the quality of implementation, which is 
critical to maximizing the program’s intended benefits and demonstrating strategy effectiveness. 
Process evaluation also provides the information needed to make adjustments to strategy 
implementation to strengthen effectiveness. 

Specifically, process evaluation can be used to: 

• Paint a clear and compelling picture of the population targeted with each strategy 

• Reach important target audiences of stakeholders 

• Provide data for program improvement efforts 

• Distribute the information through as many channels as possible to reach target audience 


Why Not Just Look at Results? 


Outcome evaluation looks at results. It measures the direct effects of program activities on 
targeted recipients, such as the degree to which a program increased knowledge about the use 
of alcohol, tobacco, and other drugs. But results don’t tell the whole story. Evaluation that only 
focuses on outcomes is sometimes called a “black box” evaluation because it does not take 
process evaluation into consideration. 

Disappointing outcome evaluation results can frequently be illuminated by examining how the 
program was implemented, the number of clients served, dropout rates, and how clients 
experienced the program. Those same kinds of questions can also explain positive evaluation 
results. (You can’t take credit for positive results if you can’t show what caused them.) 

Outcome evaluation alone, without a process evaluation component, won’t provide information 
about why a program did or didn’t work. 
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Defining Quality 


Quality implementation means that the implementers of each strategy 
have: 

• Assured the strategy matches the cultural, developmental, and gender characteristics of 
the population 

• Received training or technical assistance to support appropriate implementation of the 
intervention 

• Worked with the program developer, policy expert, or the evaluator to understand core 
components—the elements most responsible for demonstrated outcomes 

• Assessed the need for any adaptations to the strategy, especially core components, in 
order to meet the particular needs of the target population 

• Sought input from the program developer about planned adaptations to assure they are 
consistent with the core components 

• Planned necessary adaptations to target population, program content or materials, delivery 
setting or timeframe to assure integrity of implementation 

• Sought to deliver program’s core components with fidelity when possible 

• Tracked implementation through process evaluation as well as all planned and 
unanticipated adaptations to inform outcome evaluation findings 

• Used process evaluation data to inform and strengthen implementation when outcome 
evaluation did not reveal desired program results 

Process evaluation measures should be designed to assess how well the implementers 
adhered to those items. 


What to Do with Disappointing Process Data 


If process data suggest the program was not effective, practitioners should work with 
project stakeholders to: 

• Determine if the program was implemented with quality 

• Determine if strategy was appropriate for the population’s needs (review needs assessment 
data and program theory) 

• Review evaluation strategies and measures to ensure they are appropriate and valid 
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